top of page
  • Writer's pictureChockalingam Muthian

LLM Tech Stack

Pre-trained AI models represent the most important architectural change in software development. They make it possible for individual developers to build incredible AI apps, in a matter of days, that surpass supervised machine learning projects that took big teams months to build.


Workflow is into 3 stages:


Data preprocessing / embedding: This stage involves storing private data (legal documents, in our example) to be retrieved later. Typically, the documents are broken into chunks, passed through an embedding model, then stored in a specialized database called a vector database.


Prompt construction / retrieval: When a user submits a query (a legal question, in this case), the application constructs a series of prompts to submit to the language model. A compiled prompt typically combines a prompt template hard-coded by the developer; examples of valid outputs called few-shot examples; any necessary information retrieved from external APIs; and a set of relevant documents retrieved from the vector database.


Prompt execution / inference: Once the prompts have been compiled, they are submitted to a pre-trained LLM for inference—including both proprietary model APIs and open-source or self-trained models. Some developers also add operational systems like logging, caching, and validation at this stage.


You can find more details in the following link - https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/


29 views0 comments

Recent Posts

See All

Modern Data Platforms don’t do ETL Anymore

Modern data platforms have changed dramatically in recent years. Classical planned traditional on-premise Data Warehouses are increasingly shifting towards more cloud-based Data Lakes and Lakehouses.

Comments


bottom of page