Drugiego dnia przychodzi w pracy mail tak straszno-śmieszny, że wart uwiecznienia. Z obrazkiem.
(Przyjmijmy, że aplikacja z dostępem do LLM-ów, przy której pracujesz, nazywa się Tatanka).
(Przyjmijmy, że aplikacja z dostępem do LLM-ów, przy której pracujesz, nazywa się Tatanka).
Hi (...) and Tatanka Dev Team,
We trying to understand condition and investigating options to increase accuracy and reliability of generated output for a number of use cases.
Primary use case is;
User can querying sets data and the output is highly accurate (~90+%) and highly repeatable.
Use cases applicable to;
- Financial calculations and projections
- Product portfolio risk assessment (+ status changes)
- Supply chain risk assessment (+ status changes)
1. Where are we today on our system's accuracy and reliability?
2. What are our plans for increasing accuracy of Tatanka?
- Are we greater than 50% accuracy using current method of vectorRAG?
- See attached example vectorRAG vs GraphRAG
- What are the conditions (system and usages requirements) that ensures higher accuracy with greater reliability?
- What is our current benchmark by % of accuracy achievable, and what is our next target of accuracy %?
- What are the conditions (requirements, behaviors) that ensures higher accuracy with greater reliability?
- What will it take for our system to get 99.9% accuracy with high degree reliability?
3. How are we factoring knowledge graphs, GraphRAG, and other methods/models towards increasing both accuracy and reliability?
4. If in the meantime, interested in examples and guidance;
- to articulate what the accuracy limits that currently exist
- to articulate what conditions contribute to maximizing accuracy for the current capabilities of Tatanka (to help inform best practices).
