Here is a quick guide to run your own local large language model, that has been fined tuned with the alpaca instructions on google colab. The model is called “flan-alpaca-large”.

First set the runtime to GPU. Then run this code, you will need to uncoment the !pip install transformer call the first time you run it.

# Install the transformers library first!
# !pip install transformers

from transformers import pipeline

# define the model name
model_name = "declare-lab/flan-alpaca-large"

# load pipeline
model = pipeline(model=model_name)

After the model is loaded, you will need to define a prompt and generate predictions.

# Define a prompt
prompt = """Instruction: Give me a list of the most likely 5 causes of headache for adults."""

# generate predictions
yhat = model(prompt, max_length=128, do_sample=True)

# Print the output
yhat[0]['generated_text']

Model Output:

The most likely 5 causes of headache for adults are overwork, anxiety, overtired, and unhealthy lifestyle choices. Overwork is caused by high levels of stress, which can disrupt sleep patterns, and anxiety can be related to recent changes in your medical history or lifestyle. Overage or unhealthy lifestyle choices can lead to headaches, as well as the flu or other respiratory illnesses.

Hope you enjoyed this short tutorial on how to run your own local LLM.

Check out my youtube channel for more coding tutorials.


Leave a Reply

Your email address will not be published. Required fields are marked *