Skip to main content

Data Science: Tools

VIM-MLFLOW PLUGIN TO BROWSE MLFLOW RESULTS IN VIM vim-mlflow-zoom-screenshot
I love MLflow a lot, and I use it all the time for both work and personal projects. There are projects in which I go all-out and auto-log extensive graphical artifacts to better enable modeling improvements - like with example images for the different error cases, and ROC curves, and so on. And for those it's really nice to use its website as a one-stop UI to where those artifacts are logged and reviewed by others. But there are also a lot of projects where I really don't log much more than the modeling parameters and performance metrics, just numbers, and yet I'm still leaving my terminal-based workflow in Vim and looking up and comparing those numbers on the (in this case needlessly point-and-click) MLflow website. So I wrote this Vim plugin...
POSTGRESQL+SCHEMASPY VIA DOCKER CONTAINERS database relational diagram
When prototyping a new database from its initializing SQL code, and wanting to examine details of its relational architecture or demonstrate it to others, it's really convenient to simply pull my aganse/quick_postgresql repo. I just pop in my table creation lines in init.sql, make run, and view the interactive SchemaSpy website generated at its internal port. As the database sticks around and grows into use, still it's really helpful as it evolves to be able to keep rerunning and updating that SchemaSpy website as a standard part of the workgroup toolset.
INTERACTIVE GPS DATA VISUALIZATIONS IN PYTHON/JUPYTER gpsmap
Did you know you can plot your geographic data on interactive maps embedded directly in your Python notebooks? Check it out, as we play with and analyze some GPS tracking data. A database of tracked walking routes data available on a health/fitness website provides a convenient trove of data not only to play with, but also to explore the geometric interference effects of downtown buildings upon GPS track solutions.
GETTING MLFLOW+DATABASE RUNNING QUICKLY VIA DOCKER mlflow_screen_shot
This provides a get-running-quickly Docker-compose setup using containers for MLflow, PostgreSQL, and NGINX. By default it’s fully self-contained, with database data and artifact files kept in their own Docker volumes. Alternate compose files and a few environment variables let you instead point MLflow at a pre-existing database such as AWS RDS, or store artifacts like models and plots in the local filesystem or an S3 bucket. Simple make commands start and stop everything, and can populate it with a small example project for testing. We'll run MLflow's database in PostgreSQL, and put an NGINX reverse proxy in front of the MLflow website to allow some level of access restriction (say for a workgroup within an already-firewalled company intranet).
MULTI-PHASE LINEAR REGRESSION MPLR
A routine to fit multiple co-joined straight lines to a set of data points. Note the popular name (above, or sometimes "segmented linear regression") for this topic may be a bit misleading, as this is in fact a nonlinear regression problem due to solving for the intersection points of the co-joined lines. We also explore the statistics of the solution to this nonlinear problem, and provide a Monte Carlo simulation to estimate the uncertainty of the solution parameters.
No matching items