Develop an AI/ML-driven Chatbot which is Ministry Specific to help the Citizens to resolve their common queries related to filing a Grievance in the CPGRAMS portal (https://pgportal.gov.in) and expedite smooth submission of grievances.[1]
Embeddings are created using langchain and CSVLoader
loader = CSVLoader(file_path='darpg.csv', source_column="Code")
documents = loader.load()| Column | Non-Null Count | Dtype |
|---|---|---|
| Code | 18246 | int64 |
| Description | 18246 | object |
| OrgCode | 18246 | object |
| Parent | 18155 | float64 |
| Stage | 18246 | int64 |
| Destination | 10067 | object |
from langchain_community.embeddings import HuggingFaceBgeEmbeddings
model_name = "mixedbread-ai/mxbai-embed-large-v1"
model_kwargs = {"device": "cuda"}
encode_kwargs = {"normalize_embeddings": True}
hf = HuggingFaceBgeEmbeddings(
model_name=model_name, model_kwargs=model_kwargs, encode_kwargs=encode_kwargs
)url = "<URL>"
api_key = "<API"
qdrant = Qdrant.from_documents(
docs,
hf,
url=url,
prefer_grpc=True,
api_key=api_key,
collection_name="darpg-knowledge-base",
)The entire code is shared on github at https://github.com/kokamkarsahil/darpg/tree/main/Problem Statement 2 and the API is complaint with OpenAPI
The demo uses phi-2 which can be easily interchanged with different model or a hosted infernce endpoint
FROM python:3.11-slim
RUN pip install poetry==1.6.1
RUN poetry config virtualenvs.create false
WORKDIR /code
COPY ./pyproject.toml ./README.md ./poetry.lock* ./
COPY ./package[s] ./packages
RUN poetry install --no-interaction --no-ansi --no-root
COPY ./app ./app
RUN poetry install --no-interaction --no-ansi
EXPOSE 8080
CMD exec uvicorn app.server:app --host 0.0.0.0 --port 8080