Skip to content

Query

This module provides a command line interface for querying a language model using a LLM and a Faiss index of embeddings containing knowledge. It uses the Llama library to interact with the language model and the Faiss library to create and search the index.

Note

If an index_path is provided, the function loads the Faiss index and searches it for the closest embeddings to the question embedding. It then prompts the user with a message that includes the context of the closest embedding and the original question.

prompt_with_context()

Returns a prompt to request relevant text to answer a given question.

Returns:

Name Type Description
str str

A string prompt that includes the given context and question, and asks for relevant text.
Source code in src/query.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
def prompt_with_context() -> str:
    """
    Returns a prompt to request relevant text to answer a given question.

    Returns:
        str: A string prompt that includes the given context and question, and asks for relevant text.
    """
    return """
    Use the following portion of a long document to see if any of the text is relevant to answer the question.

    Context: {context}
    Question: {question}
    Provide all relevant text to the question verbatim. If nothing relevant return "I do not know" and stop answering.
    Answer:"""

query(question=typer.Argument(Ellipsis, help='Question to answer.'), model_path=typer.Argument(Ellipsis, help='Folder containing the model.'), index_path=typer.Argument(None, help='Folder containing the vector store with the embeddings. If none provided, only LLM is used.'))

Ask a question to a Language Model (LLM) using an index of embeddings containing the knowledge.

If no index_path is specified, it will only use the LLM to answer the question. Otherwise, it will use the embeddings in the index_path to find relevant text before prompting for an answer.

Parameters:

Name Type Description Default
question str

The question to answer.

 typer.Argument(Ellipsis, help='Question to answer.')
model_path str

The folder containing the LLM model.

 typer.Argument(Ellipsis, help='Folder containing the model.')
index_path Optional[Path]

The folder containing the vector store with the embeddings. If none provided,

 typer.Argument(None, help='Folder containing the vector store with the embeddings. If none provided, only LLM is used.')

Returns:

Name Type Description
None

The response will be printed to the console.
Source code in src/query.py 
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def query(
    question: str = typer.Argument(..., help="Question to answer."),
    model_path: str = typer.Argument(..., help="Folder containing the model."),
    index_path: Optional[Path] = typer.Argument(
        None,
        help="Folder containing the vector store with the embeddings. If "
        "none provided, only LLM is used.",
    ),
):
    """
    Ask a question to a Language Model (LLM) using an index of embeddings containing the knowledge.

    If no `index_path` is specified, it will only use the LLM to answer the question. Otherwise, it will use the
    embeddings in the `index_path` to find relevant text before prompting for an answer.

    Args:
        question (str): The question to answer.
        model_path (str): The folder containing the LLM model.
        index_path (Optional[Path]): The folder containing the vector store with the embeddings. If none provided,
        only the LLM will be used.

    Returns:
        None: The response will be printed to the console.
    """
    global llama
    if not llama:
        if not model_path:
            raise Exception(
                "You need to specify the model path the first time you ask a question"
            )
        llama = Llama(model_path=model_path, verbose=False, embedding=True)
    if index_path:
        index = FaissIndex()
        index.load(index_path)
        embedded_question = np.array([llama.embed(question)])
        context = index.search(embedded_question)
        question = prompt_with_context().format(context=context[0], question=question)
    response = llama(prompt=question)["choices"][0]["text"]
    print(response.split('"""')[0])