Templates: Document QnA
Want use LLMs to query or summarize documents or other information privately and securely? The Document QnA is the fastest way to get started. Follow the steps below to upload your documents and get going.
- Open the Document QnA Element settings and make the following adjustments.
- Trained Artifact: If you have previously trained a custom LLM you can select it here. It will override the Base Model Architecture and Quantization Rank settings.
- Base Model Architecture: Select any of the supported models, or leave this as the default.
- References Folder Path: Upload a folder of containing the documents you want to interact with. It can be any mix of Word Docs, PDFs, or .txt files.
Note: Even if you have only 1 document it needs to be in a folder. - Base Model Storage Path: Select where you want the base LLM model to be stored on your machine.
- Model System Prompt: This default prompts your model to act as an assistant. You can leave this setting alone, or play with it to change how the model responds.
For example: ask the model to talk like a Pirate or Rock Star, speak in a casual tone, or speak in like a poet. - Leave all other settings as default.
- Press Run
- Start chatting!
If you want a deeper dive into all of the settings to really customize your Document QnA flow, read our guide here.