Skip to main content

Data Science: LLMs

RAG EVALUATION WITH MLFLOW + LANGCHAIN panel
If you've seen any of my other DS/ML content you know I'm a big fan of MLflow. The bottom line is that making a new model is typically the quick and easy part. It's figuring out how to make it consistent and how to improve it that are the hard part - and the key is being fastidious and objective about what you (and your team) know about the model and data that you're working with. This is equally true with GenAI language models and RAG as it is for classic machine learning and deep learning. MLflow 3 has an expansive feature set for GenAI development and quantifiable evaluation that provides a lot of the toolset straight out of the box (or "container" 🙂?).
SEMANTIC SQL EXPERIMENTATION WITH POSTGRESQL'S PGVECTOR EXTENSION semanticsql
For some types of semantic analysis we don't always need to always pull big dataframes of data out of the database to do the analysis externally if we already have embedding vectors stored in the database. PostgreSQL's pgvector extension provides powerful functionality for semantic search and aggregation right there in the SQL queries. Let's experiment with some of this by grabbing an arbitrary dataset (how about Kaggle's Seattle AirBnB listings/reviews), popping that into a database, running an embedding model on that to create embedding vectors, and exploring SQL queries using this functionality.
GPT_CLIENT CLI WITH PARAMETER CONTROL, WEBLINK SUBMISSION, & SYNTAX HIGHLIGHTING GPT example
(This project is old but I find I'm using it again lately for a quick CLI way to test responses from different API model versions while injecting web content as I hone RAG apps.) I have found OpenAI's GPT models to be incredibly productive tools and use them often in my work now. But to get what I want out of the models for my uses has taken accessing the models from the API rather than the ChatGPT website GUI. This allows me to set the model, change some of the model parameters, format the output as I wish, and run the whole thing in my terminal. Of course the process of making the app has been useful education to understand how the models work as well, including how interacting with them via API enables other use cases from other automated code.
No matching items