ollama

ollama is a CLI tool that enables users to utilize and run different large language models (LLMs) offline on local machines.

It offers a library of pre-built open source models such as Aya, Llama 3, Phi-3, Mistral, Mixtral, Gemma, Command-R and many more. All models are ready for use, download, and customize, each differing in parameters and sizes.

By using the CLI, it provides extensive customization options, including command prompts to customize a model and system messages tailored for unique interactions.

ollama is cross platform on Windows, macOS and Linux and is beneficial for research engineers, software engineers and hobbyists looking to leverage LLMs without relying on cloud-based solutions.

What is ollama?

How to install ollama?

To install ollama, you can either go onto ollama's installation page or you can install ollama quickly for your platform with different package managers on Terminal Trove.

What LLM models does ollama support?

The ollama models in the model library are free to use and are open models such as Llama (llama2, llama3, codellama), Gemma (gemma, codegemma), Mixtral, Phi-3, Aya, Command-R, Qwen and other custom models.

Where does ollama store models?

Windows

C:\Users\<username>.ollama\models

macOS

~/.ollama/models

Linux

/usr/share/ollama/.ollama/models

How to use ollama?

Below are the 5 most common commands to use when working with ollama:

1. Downloading LLM models

Download LLM models such as Llama 3 to your machine, you can specify the model size with the colon (':')

ollama pull llama3

ollama run llama3:70b

2. Running LLM models

Run LLM models on your machine with ollama.

ollama run llama3

3. Listing LLM models

List all the installed LLM models downloaded with ollama.

ollama list

4. Copy LLM models

Copying LLM models for use with a custom model with ollama.

ollama cp llama3 llama-3-copy

5. Remove LLM models

Removing LLM models with ollama.

ollama rm llama3