Off-the-shelf LLMs are excellent at manipulating and generating text, but they only know general facts about the world and probably very little about your use case. Retrieval augmented generation (RAG) refers not to a single algorithm, but rather a broad approach to provide relevant context to an LLM. As industry applications mature, RAG strategies will be tailored case-by-case to optimize relevance, business outcomes, and operational concerns.

crosspost from https://blog.dagworks.io/p/retrieval-augmented-generation-reference-arch

Skip learning convoluted LLM-specific frameworks and write your first LLM application using regular Python functions and Hamilton! In this post, we’ll present a containerized PDF summarizer powered by the OpenAI API. Its flow is encoded in Hamilton, which the FastAPI backend runs and exposes as an inference endpoint. The lightweight frontend uses Streamlit and exercises the backend. (GitHub repo)

crosspost from https://blog.dagworks.io/p/containerized-pdf-summarizer-with

Are you using Feast? or perhaps you are having trouble with it? Or perhaps you are considering adopting it? In this post, you will learn the operational benefits of using Feast with Hamilton. Feast will act as your “feature store”, while Hamilton will be your in-process “feature processing engine”. We start by providing an overview of Feast, then show how Hamilton fits into the picture. At the end, we give a recipe for where to go from here, depending on where you are in your “feature journey”.

crosspost from https://blog.dagworks.io/p/featurization-integrating-hamilton

This post will show you how to use Hamilton with Prefect, two open source projects. At a high level, Prefect orchestrates where/when/what/how code runs (think macro) and Hamilton helps author clean and maintainable code for data transformations (think micro).

crosspost from https://blog.dagworks.io/p/simplify-prefect-workflow-creation

In this post, we’re going to share how Hamilton can help you write modular and maintainable code for your large language model (LLM) application stack. Hamilton is great for describing any type of dataflow, which is exactly what you’re doing when building an LLM powered application. With Hamilton you get strong software maintenance ergonomics, with the added benefit of being able to easily swap and evaluate different providers/implementations for components of your application.

crosspost from https://blog.dagworks.io/p/building-a-maintainable-and-modular

This post walks you through the benefits of having Hamilton and Airflow directed acyclic graphs (DAGs) work in tandem. Airflow is responsible for orchestration (think macro) and Hamilton helps author clean and maintainable data transformations (think micro). The reason you can run Hamilton with Airflow, is that Hamilton is just a library with a small dependency footprint, so one can get started with Hamilton in no time! For those that are unfamiliar with Hamilton, we point you to an interactive overview on tryhamilton.dev. Otherwise we will talk about Hamilton at a high level and point to reference documentation for more details.

crosspost from https://blog.dagworks.io/p/supercharge-your-airflow-dag-with

Hamilton is an open-source Python microframework developed at Stitch Fix. It automagically organizes Python functions into a directed acyclic graph (DAG) from their name and type annotations. It was originally created to facilitate working with tabular datasets containing hundreds of columns, but it’s general enough to enable many data science or machine learning (ML) scenarios.

crosspost from https://medium.com/@thijean/the-perks-of-creating-dataflows-with-hamilton-36e8c56dd2a