Ollama Tutorial: How to Run LLM Locally

This guide covers how to use Ollama for running large language models (LLMs) locally on your machine. Ollama simplifies the process of hosting and interacting with open-source LLMs, empowering users to achieve privacy-focused, offline AI capabilities while maintaining control over their workflows. Follow this step-by-step tutorial to install Ollama, download models, test them, and explore advanced features such as API integrations and agentic coding.

Prerequisites for Running LLM Locally

Before beginning, ensure you meet these requirements:

prerequisites

Your machine must meet the minimum hardware requirements. Commonly, you'll need at least 16GB RAM for smaller models and 64GB+ for large, high-performance models.
Access to a terminal or command-line interface.
A stable internet connection to download models.
Optionally, familiarity with tools like Docker or command-line programs may help.

Steps to Install and Run Ollama

Setting up Ollama is straightforward. Follow the steps below to install the tool and get it running on your machine.

steps

Download Ollama
Visit ollama.com and locate the installation instructions. For a direct installation on macOS, use the following command:
bash
```
brew install ollama/tap/ollama
```
Install Ollama
Once the download is complete, run the following command in your terminal:
bash
```
ollama
```
This starts the Ollama service and verifies that it is installed correctly.
Verify the Server Functionality
Test the server by issuing an API call to the local server. Use the following curl command to check if the API is running:

bash

   curl http://localhost:11434/api/status  
   # Expected output: {"up":true}

If the response confirms that the server is up, you are ready to proceed.

Downloading and Testing a Local Model

This section demonstrates how to download, load, and test an LLM using Ollama.

steps

Select and Download a Model
Choose a model that fits your hardware specifications. For a smaller model suitable for a laptop with 16GB of RAM, use this command:

bash

   ollama pull llama2:3b
   # Output: Retrieval and download of the model files for Llama 2 3B

List Available Models
Verify the downloaded model is accessible. Use:

bash

   ollama list
   # Lists available local models, such as Llama 2 3B

Interact with the Model
Test the model's response with small tasks like answering questions or writing code:

bash

   ollama run llama2:3b
   # Input: "Write a JavaScript function to reverse a string."
   # Expected output: "function reverseString(str) { return str.split('').reverse().join(''); }"

Exploring Advanced Features and API Access

Ollama supports advanced capabilities such as providing API access for integration and agentic workflow support. Here's how you can get started.

steps

Run Ollama in Server Mode
Start the Ollama server in debugging mode to monitor API requests and responses:

bash

   ollama serve
   # Server starts outputting logs with incoming API requests and processed responses

Test the API with curl
Use the curl command to test interactions with the running Ollama instance:

bash

   curl -X POST http://localhost:11434/api/generate -d '{"model": "llama2:3b", "prompt": "Explain the concept of recursion."}' -H "Content-Type: application/json"
   # Expected JSON response with the model's explanation of recursion

Integrate with Agentic Tools
To utilize advanced agentic tasks, consider integrating with tools like OpenCode. First, install the OpenCode CLI:

bash

   curl -L https://opencode.example.com/install.sh | bash

Launch OpenCode and link it to an Ollama model:

bash

   ollama launch opencode --model=llama2:3b
   # This connects the OpenCode harness to your local Ollama instance and the selected LLM.

Test agentic workflows by providing task-oriented prompts, such as automating code generation or fetching system information.

Explore Additional Models
Review other models available on ollama.com by searching for keywords:

bash

   ollama search Qwen

Download a model by specifying its name and parameter size:

bash

   ollama pull qwen:2.5-coder
   # Downloads Qwen 2.5 Coder model

FAQ

Can I run Ollama on a machine with less than 16GB RAM?

While possible, running LLMs on a machine with less than 16GB of RAM will limit the size and performance of the models you can use. Smaller models with fewer parameters (e.g., 2–3 billion) may still work, but the system may experience memory constraints.

How do I choose the right model for my hardware?

The model you choose depends on your hardware's RAM capacity. For machines with 16GB or less, stick to smaller models (3B parameters or fewer). For more robust systems with 64GB+ RAM, you can run larger models with up to 13B or higher parameters.

How do I update Ollama to the latest version?

You can update Ollama using the same package manager you used for installation. For example, if you installed Ollama using Homebrew on macOS, run the following command:

bash

brew upgrade ollama

Does Ollama require an internet connection?

An internet connection is required to download models initially. Once models are downloaded, Ollama can operate fully offline for local model inference.

Related guides

Ollama Tutorial: How to Run LLM Locally

Prerequisites for Running LLM Locally

Steps to Install and Run Ollama

Downloading and Testing a Local Model

Exploring Advanced Features and API Access

More guides

Using ChatGPT for Developers: A Coding Workflow

MCP Streamable HTTP: How to Migrate from Deprecated SSE

Setting Up Hooks in Claude Code for Reliable Workflow Automation