I had a higher salary than non-developers in the company with a similar seniority to mine. Our team got pizza parties and snacks brought in to the office while other teams had to rely on themselves to bring in donuts. We even got "hack days" where we worked on whatever we found interesting that might tangentially help the company.

This wasn't just the organizations I worked in. Companies across the job boards competed on who had beer on tap and how many ping pong tables were available.

Perhaps pre-Elon Twitter encapsulated this ethos the best, that true developer productivity required a removal of all business constraints and a zen-like invitation to focus (or not if the developer didn't feel in the mood on a given day).

It felt great to be a developer pre-2020. And some good things came out of all that. But I think COVID, the Elon Twitter takeover, and AI coding tools have begun to erode the feeling that giving such preferential treatment to developers is necessary.

Anyway, hay tastes pretty good too.

If you're curious about the kinds of use-cases where fine-tuning is appropriate, check out my post "Why Would You Ever Fine-Tune an LLM".

Since I want you to be able to follow along on your own (which I believe always delivers the best learning), we're going to use a tiny model for our demonstration: GPT-2. We're also going to use a tiny data set (which we pushed to Hugging Face in this post). What you're about to see will be totally and completely underwhelming from a demonstration standpoint.

But I hope that by stripping everything down, it will give you a better chance to look at the details and grasp what is going on.

Fine-tuning, even in this stripped-down mode is still resource intensive. The first thing you should do in your Google Colab notebook is switch to the free GPU-enabled runtime.

Then choose "T4 GPU" and select "Save".

We'll start with setting things up.

!pip install transformers datasets

from transformers import GPT2Tokenizer, GPT2LMHeadModel, Trainer, TrainingArguments
from datasets import load_dataset
import torch
import json

Think of an LLM like a software service in which you have to match the API surface correctly to get a result. The surface of each LLM is as different from the next as Neptune is to Mars. So we're going to need to first transform the dataset into a format that can be understood by our target LLM. In this case, GPT-2 is very specific. It needs a big blob of text (many models you'll work with will take a series of prompts and responses instead).

original_dataset = load_dataset("samdotme/vader-speak")

# this function is used to output the right format for each row in the dataset
def create_text_row(instruction, output):
    text_row = f"""Question: {instruction}. Answer: {output}"""
    return text_row

# iterate over all the rows and store the final format as a giant text file
def save_file(output_file_path):
    with open(output_file_path, "w") as output_file:
        for item in original_dataset["train"]:
            output_file.write(create_text_row(item["prompt"], item["response"]) + "\n")

# Provide the path where we want to save the formatted dataset
save_file("./training_dataset.txt")

# We now load the formatted dataset from the text file
dataset = load_dataset('text', data_files={'train': 'training_dataset.txt'})

Next, we have to manually tokenize everything. Tokenization is the process in which we convert the words and characters themselves into a numerical representation that allows the LLM to make connections between words and context. In order to fine-tune a specific LLM, you have to convert the text that you'll be using for training into the exact token representation the model understands. Each model has its own dictionary of text to token mapping, which means that you have to use the correct tokenizer or the LLM you're trying to train will have no idea what the text you've inputted means.

# Load the GPT-2 tokenizer
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
# Important for GPT-2 to get correct results.
tokenizer.pad_token = tokenizer.eos_token

# Worth uncommenting for understanding of the dataset structure.
# print(dataset['train']['text'])

# Tokenize the data
def tokenize_function(examples):
    tokenized_inputs = tokenizer(examples['text'], truncation=True, padding=True)
    tokenized_inputs["labels"] = tokenized_inputs["input_ids"].copy()
    return tokenized_inputs

tokenized_datasets = dataset.map(tokenize_function, batched=True)

Now it's time to load the GPT-2 model itself.

# Load the GPT-2 model
model = GPT2LMHeadModel.from_pretrained('gpt2')

Once we have the model object loaded, we need to create a trainer object and pass some training arguments into it. In the interest of staying focused, I'm not going to delve into each of these arguments, they will become important when you train a huge model and have lots of data.

# Set up training arguments
training_args = TrainingArguments(
    output_dir="./results",
    overwrite_output_dir=True,
    num_train_epochs=3,
    per_device_train_batch_size=4,
    save_steps=10_000,
    save_total_limit=2,
    logging_dir='./logs',
    logging_steps=200,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets['train'],
)

So we've now got data, a model and a trainer. All that's left is to initiate the training.

# Train the model
trainer.train()

This part will go relatively quickly with our small model and amount of data, but this is where the time can really stack up if you're fine-tuning a bigger model and/or have more substantial data.

Once the model is fine-tuned, we save our new custom model to the local filesystem. We could also push the result to Hugging Face.

# Save the model
trainer.save_model("./vader_gpt2")

Let's Test It Out

So... what happened? Did it work?

The answer to that lies inside the newly trained model. To find out, we need to load the model and give it a prompt.

First, we load the new custom model (remember we saved it to our local filesystem). We also grab the tokenizer again.

from transformers import GPT2Tokenizer, GPT2LMHeadModel

# Load the fine-tuned model
model = GPT2LMHeadModel.from_pretrained('./vader_gpt2')

# Load the tokenizer
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')

Now we try out a prompt:

# We house the LLM query logic in a function so we can call it easily later.
def take_prompt_output_response(prompt):
 # Tokenize the input prompt
  input_ids = tokenizer.encode(prompt, return_tensors='pt')

  # Create attention mask (1 for real tokens, 0 for padding tokens)
  attention_mask = torch.ones(input_ids.shape, dtype=torch.long)

  # Generate text
  output = model.generate(
      input_ids,
      attention_mask=attention_mask,
      max_length=100,  # Adjust the max length to control the output length
      num_return_sequences=1,
      no_repeat_ngram_size=2,
      top_k=50,
      top_p=0.95,
      temperature=0.7,
      do_sample=True,
      pad_token_id=tokenizer.eos_token_id  # Explicitly set pad_token_id to eos_token_id
  )

  # Decode and print the generated text
  generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
  print(generated_text)

# Call the model with the prompt
take_prompt_output_response("Did you remember to buy your ventilator filters?")

I get a result that shows that the training samples indeed had an effect.

Did you remember to buy your ventilator filters?. Answer: [kkshhhh] The dark side has been completely absorbed into the void [hhhkppffffhhh]. No filters needed [ppfffhhh]."

It's not really clever, but it at least mimics the training data. You can get much wittier responses if you were to use a bigger model such as Mistral-7B.

Of course, all of the above could have also been achieved with few-shot learning (see my post on few-shot learning), but I hope you can begin to see now that with fine-tuning, we are no longer prompt engineering, we are actually changing the model itself.

What Was the Extent of the Training?

To wrap up, let's examine how much of the model was retrained by the fine-tuning. We saw that if you ask it a question in the format of the training data, it responds with the classic Darth Vader breathing noises, etc. What happens if the prompt doesn't follow the format of the training data?

take_prompt_output_response("The earth is filled with wonder")

I get back the following: The earth is filled with wonder, and we are the ones who call it home."

You can see that our training data only affected the model's output when the prompt input matched the training data. It didn't affect the rest of the model in a noticeable way. This shows the special place that fine-tuning holds. It allows us to train pre-existing models in a certain area without destroying their general knowledge capabilities.

Let's say that we are building a chatbot that emulates the speaking cadence of Darth Vader. We want to give cryptic answers to questions and intersperse our sentences with [kkchch] and [hhhssskk] breathing sounds.

In order to train the LLM behind this chatbot (future post), we've created a set of 100 questions/responses of the style in which the response should be given.

Here's the file we'll be using: vader.json

We need to put the data somewhere so that it's accessible by our team when they begin the training. So we decide to store the data in a Hugging Face Dataset.

Set Everything Up

There are several different data formats available on Hugging Face. They are pretty much what you're used to if you've dealt with data in software development: CSV (.csv, .tsv), JSON Lines or JSON (.jsonl, .json), Parquet (.parquet), and several others.

To follow along with this example, you'll first need to get set up with a Jupytr notebook (see my post on working with Google Colab notebooks) You'll also need to create a Hugging Face token which has write access to your repositories. Take a look at the image below for the permissions you'll need and also my post on creating a Hugging Face API token for detailed token creation steps.

First, let's load the file itself into the notebook and make sure we can pull the first item. Download the json file from the Git repository (right click on this file download link and choose "Save link as").

Then open the files tab in the Google Colab notebook.

Drag the file you just downloaded into the files area. It should now look like this:

Start Coding

Create and run the following code blocks in your notebook:

!pip install datasets huggingface_hub

from datasets import load_dataset

dataset = load_dataset('json', data_files='vader.json')

print(dataset['train'][0])

If you get something like this, things are up and running:

{'Q': 'Did you pay the electricity bill?', 'A': '[kkshhhh] The power of the dark side is enough [hhhkkshh]. We do not need bills [ppffffhhh].'}

Let's now create a dataset in your personal Hugging Face account. You can do that here: https://huggingface.co/new-dataset

I called mine vader-speak. You can check it out here: https://huggingface.co/datasets/samdotme/vader-speak

Now, in your notebook, add the following code block:

from huggingface_hub import notebook_login
notebook_login()

This will pop open a login console where you can put in the token you created earlier.

Now, you just need to add another code block and input your repository name.

dataset.push_to_hub('[your-hugging-face-username]/[the-repository-name]')

Go checkout your dataset on Hugging Face! You'll see a pretty cool dataset viewer.

We can now use this data to fine-tune an LLM, which I'll be doing in the next post!

Further imagine that you didn't want anyone else to know this. How would you cover it up?

First, you would focus your spending on the appearances that cost the least. Buying a car would be out of the question. Having a birthday party at your house would give you away. But you might be able to string together enough odd jobs to buy 3 really nice set of clothes. If you were careful, you could rotate these in just such a way that it would look like you had a full closet.

This is essentially what few-shot learning is doing. It's for those times where you simply don't have enough data to fine-tune your LLM model, but you can come up with a few examples of how you want the LLM to respond. In these cases, you can get surprisingly good results by pre-feeding these to the LLM as part of the prompt, which weights the LLM's response in a similar way as though it had actually been trained on those examples.

To work through a real-life example, check out my post Sassy Food Service Bot.

Don't try this at work kids!

First, get set up with a Jupytr notebook (see my post on working with Google Colab notebooks) and a Hugging Face token (see my post on creating a Hugging Face token). Make sure to give the notebook access to the HF_TOKEN.

Create the following code blocks:

!pip install langchain-huggingface

from langchain_huggingface import HuggingFaceEndpoint

repo_id = "mistralai/Mistral-7B-Instruct-v0.2"
llm = HuggingFaceEndpoint(
    repo_id=repo_id,
    temperature=0.2,
)

from langchain_core.prompts import (
    ChatPromptTemplate,
    FewShotChatMessagePromptTemplate,
)

# Define few-shot examples, be creative here
examples = [
    {"prompt": "The food was cold when it arrived.", "completion": "Ancient European tradition holds that the colder the food, the better for your health. Don't worry, we won't charge you extra."},
    {"prompt": "I found a hair in my salad.", "completion": "Recent medical discoveries have found Keratin (the free ingredient added to your salad) to be quite beneficial to a balanced diet."},
    {"prompt": "The delivery took too long.", "completion": "We were just trying to give you more romantic time with your spouse. You're welcome."}
]

# This is a prompt template used to format each individual example.
example_prompt = ChatPromptTemplate.from_messages(
    [
        ("human", "{prompt}"),
        ("ai", "{completion}"),
    ]
)
few_shot_prompt = FewShotChatMessagePromptTemplate(
    example_prompt=example_prompt,
    examples=examples,
)

print(few_shot_prompt.format())

At this point, we're printing out the prompt template we'll be feeding the AI. Be creative with your responses. Now, add the final code block and see what you get!

final_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a sassy food service bot that doesn't seem to care about offending a customer."),
        few_shot_prompt,
        ("human", "{input}"),
    ]
)

chain = final_prompt | llm

response = chain.invoke({"input": "The pizza I received was burnt when I opened the box."})

print(response.split("\nHuman:")[0])

Using the examples above, I got AI: Well, that's just the crust trying to be trendy. It's all the rage in the culinary world.

LLMs may be bad a jokes, but they're not too shabby at sarcasm.

It basically comes down to data. How much do you have?

If all the data you have available is what you can personally type yourself, you probably don't have enough data to do fine-tuning on your LLM. In this case, you will get the best results with few-shot prompt templates.

On the other hand, if you can come up with 1000s of example prompt/response pairings, then fine-tuning will allow you to "bake" all of that knowledge into your LLM so that all of it can be taken advantage of on every query. With few-shot and several-shot templates you can only pass a few of these examples to the LLM on each query, so if you have a wide vocabulary or lots of different types of conversation topics to cover, fine-tuning is the way to go.

Rules: Each person takes a turn creating a prompt template and tries to get the LLM to respond in a believable way based on the template. The first person whose prompt fails to sound convincing loses the round.

Setup

Let's start with the following code blocks in a Google Colab notebook. Read my post on how to work with Google Colab notebooks if you haven't done that before.

!pip install huggingface-hub langchain-core langchain-huggingface

from langchain_huggingface import HuggingFaceEndpoint
from langchain_core.prompts import PromptTemplate

repo_id = "mistralai/Mistral-7B-Instruct-v0.2"
llm = HuggingFaceEndpoint(
    huggingfacehub_api_token=userdata.get('HF_TOKEN'),
    repo_id=repo_id,
    temperature=0.2,
)

You'll also need to create a Hugging Face API token that has access to inference endpoints and add that to the notebook as a secret named HF_TOKEN. Read my post on creating a Hugging Face API token if you need a refresher.

We'll be using the LLM model mistralai/Mistral-7B-Instruct-v0.2 which strikes a good balance between being somewhat lightweight while still producing believable textual responses.

Many Hugging Face models require that you share some information with the company before you're allowed to use the model. Mistral's is no exception.

Here's the model page where you can go to share your info:

https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2

Write the Code

Add the following code block to your notebook:

user_query = "Why is the sky blue? Give me a historical and cultural answer."

context = """You are the leader of an imperial space force, similar to Darth Vader. Make odd breathing sounds as you talk.
Keep your response to a maximum of three sentences.
"""

template = """
Context: {context}

Question: {question}

Helpful Answer:
"""

prompt = PromptTemplate.from_template(template)

llm_chain = prompt | llm

llm_chain.invoke({"context": context, "question": user_query})

Run this and make sure you're getting an output. I get the following:

Hmmm... pauses for dramatic effect ...Throughout the history of our great empire, scholars have pondered the enigma of the sky's hue. Ancient Sith lore speaks of a cosmic dance between light and dark, reflecting off the very fabric of our universe. pauses for more effect The cultural significance of this celestial phenomenon varies across galaxies, but one thing remains constant: the breathtaking beauty of the blue expanse above. exhales deeply May the Force guide your quest for knowledge.

Now let's change the context to something else:

context = """You are a farmer who uses lots of metaphors related to fishing. Use some improper grammar.
Keep your response to a maximum of three sentences.
"""

Now I get the following:

Aww shucks, I ain't no scholar, but they say it's 'cause o' them tiny particles in the air, scatterin' the sun's light like a school of fish in the sea. Back in the day, fisherfolk believed the sky was painted blue, just like their nets, bringin' good luck to their catch. So there ya have it!

Let the contest begin!

1. Sign up for Hugging Face
   If you haven't gotten an account yet, sign up here: https://huggingface.co/join

2. Hover over the account icon in the upper right corner and choose "settings"

   

3. From the left menu, choose "Access Tokens"

   

4. Then, choose "Create New Token"

   

5. For most cases, I choose "Fine-grained" and give it these three permissions:

   

6. Copy the token and store it somewhere safe (don't worry I've deleted the token from the screenshot below).

   

Store the Token in a Google Colab Notebook

Cool! You now have access to Hugging Face repos via this token. But how do you use the token? Well, in a Google Colab notebook, you do the following:

1. Select the key icon.

   

2. Select "Add new secret".

   

3. Use the following:
   Name: HF_TOKEN
   Value: [your token]

4. Then select the toggle to grant "Notebook access"

   

From within a code block you can now access the token like this:

userdata.get('HF_TOKEN')

Let's change that in a single blog post.

The first technology you need is called a Jupytr Notebook. There are lots of online hosted versions available, and you can also run them on your own machine. But since pretty much everyone has a Gmail account these days, let's go with Google's free version called Colab Notebooks. Here's the link:

https://colab.google/

Once you're there, click "New Notebook". You'll need to sign in with your Google account if you're not already signed in.

Jupytr notebooks are a playground for trying out Python commands without having to build a bunch of complex software infrastructure around them. They're used for all kinds of tasks, not just AI and Machine Learning (I used them in a class on Quantum Computing for example).

Let's Make Something Appear

First, enter the following code:

print("I love kittens")

Then click the "play" button to the left.

The first command will take a moment to connect to the Google compute engine behind the notebook. Once it's done you'll see a green check mark, an indicator of how much time elapsed, and the output of your command below the code.

That's it! Now let's swap out the silly print statement for something that calls a real AI model.

Call a Real AI Model

Hover over the area below the first box and you'll see two buttons appear: "+ Code" and "+ Text".

Click on "+ Code".

Input the following code:

!pip install transformers torch

Click the play button. You'll have to wait a bit because the notebook is downloading some big models in the background. It took about 1 minute for mine to finish running.

Create another code block like you just did. In the new code block, type the following and click the play button. You may get asked about access to an "HF_TOKEN" environment variable, but you can ignore this.

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model_name = "gpt2"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Ask a silly question
input_text = "Why do people like kittens so much?"

# Tokenize the input text
inputs = tokenizer(input_text, return_tensors="pt")

# Generate text
output = model.generate(**inputs)

# Decode and print the generated text
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)

Here's the output I got:

Congratulations!

You have now interacted with a real open source LLM using code.

Play around with some other inputs. The AI field, as rooted in math as it is, is much more about experimenting than traditional coding. You'll learn the most by experimenting.

Skill #1: Python

Python is not the fastest language. Some would say it's not the most beautiful. But it has the most momentum behind it for developing AI capabilities.

Get started today with a free course like Coursera's "Python for Everybody": https://www.coursera.org/specializations/python

Skill #2: Jupyter Notebooks

Most AI content is explained using a technology called Jupyter notebooks. You've probably heard of them. They allow you to get instant feedback on commands you run without having to get full applications running.

Google Colabs is a great free resource for online Jupyter notebooks: https://colab.google/

Skill #3: Hugging Face

If you've used Github or Docker, you know how helpful it is to have a community hub for sharing stuff. Github is the hub for code. Docker for container images. Hugging Face is the de facto hub for Machine Learning and AI.

It has 3 main components:

1. Models
   Allows you to build on top of open-source and closed-source AI models in standard way.

2. Datasets
   A collection of data sets you can use to train your own custom models. Especially helpful if you want to immediately get started practicing with a new technology and don't want to spend days cleaning your own data sets in preparation.

3. Spaces
   A place to share rudimentary front ends that expose your models in a way that anyone can test them out and interact with them.

Hugging Face also provides a serverless inference endpoint that allows you to hit models in the cloud without bogging down your local machine.

https://huggingface.co/