AI is everywhere, and companies are trying to label their products to ensure they do not get left behind in the race. This is a fair attempt, as most of the general public sees that a product using AI feels more advanced and innovative. However, we need to make sure we are implementing AI in our apps to add value rather than brag about having AI on our apps. A good understanding of how LLMs can be used will help implement them in useful ways.
The challenge is teaching old dogs(existing software written by human programmers) new tricks with minimum effort. You can replace the old dog with a new and shiny new one, but as we know, in software engineering, there are no successful rewrites.
Application Logic
Most human-written(or using AIs like Co-Pilot) applications are pre-programmed to handle expected behaviour and outcomes scenarios. For example, when a search term is entered into a search box, the logic behind the button will be programmed to call an API or directly go to a Database to get some results. This process is 100% designed and driven by humans.
Where can we use LLMs in these applications?
There are a few ways we can use LLMs.
- Single LLM call to understand the input or output
- Chain of LLM calls
- LLM is used to make about what steps to take next
- LLM is used to make decisions in a loop
- Fully autonomous approach
Approach 1: Single LLM call to understand the input or output
An LLM can be used to answer a question. This will be similar to using ChatGPT or Gemini via the UI. A user can ask a question, and then the LLM will try to provide an answer in the format that the user requested. If your application would need to use an LLM for content writing purposes, this will be the first point to start.
Approach 2: Chain of LLM calls
The following process can be followed in an RAG pipeline.
- A user sends a query in the natural language
- Generate embeddings for user input using an embedding LLM
- Query a Vector DB to retrieve similar documents
- Augment user query with retrieved documents using a text LLM
- Returns and answers in natural language
In this scenario, we are chaining multiple LLMs(can be the same type depending on the scenario) in a chain to achieve a common goal.
Approach 3: LLM is used to make about what steps to take next
This is the most straightforward approach to making an application more intelligent. Depending on the intention of a user query, an application can dynamically decide how to react. For example, suppose a user query requires additional data from an external system. In that case, the LLM should be able to help the application make that external API call by telling it which API to call and what parameters to pass in.
In general terms, this is called tool calling. The application developer can define a few tools to achieve a task or answer a query. The tool’s implementation is up to the application developer. It could be an API call to an external system or querying a database.
Approach 4: LLM is used to make decisions in a loop
In this stage, the LLMs can be called in a loop to understand what steps to take next. Depending on the output provided by the previous steps, this could be a collection of calls. An LLM will be aware of all the steps available to take, and it will decide based on the last task output. This process will continue until the LLM receives a STOP command.
At this level, the application can become unpredictable and difficult to test. However, this will help handle the scenarios that are traditionally complex to code.
Approach 5: Fully autonomous approach
Up to this point, human programmers have decided what steps an LLM can take. We can allow an LLM to figure out what step to take next and how to handle that step. However, the human programmers will have no control over the application logic, and predicting the outcomes will be nearly impossible. Some applications will require this approach to explore areas human programmers don’t understand.
This will make you wonder at which stage I can call my app that uses AI. You can do that if your application implements any of the abovementioned approaches.
Which approach is best? It depends on your use case. You should consider carefully which approach best suits a given scenario.
Do I have to build a chat interface to add AI? No, you can use LLMs to make decisions in applications that don’t need to react to a chat message. For example, you can put a prompt behind a button click for a given task with context about the user and the application’s state. LLMs can be used in the background tasks, and your user will not have any direct interactions with them.
I hope this article helps you understand better.
Leave a Reply