How to Open WebUI Ollama for Self-Hosted ChatGPT

steps

1. Set up a privileged LXC container in Proxmox.

   • Create a new LXC container with Ubuntu 20.04 LTS.
   • Mark the container as privileged and assign sufficient disk space for storing LLMs.
   • Assign a static IP to the container for predictable networking.

2. Install AMD ROCm drivers.

   • Inside the container, install AMD ROCm following AMD's documentation.
   • Verify GPU detection with:
     bashCopy

     rocm-smi
     rocm-info

3. Pass through GPU devices to the container.

   • Update the container’s configuration file in /etc/pve/lxc/<container-id>.conf to include:
     bashCopy

     lxc.cgroup2.devices.allow = c 226:* rwm
     lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir

4. Install Ollama using its installation script:

   bashCopy

   bash <(curl -s https://ollama.com/install.sh)

5. Secure GPU access for the Ollama service.

   • Adjust the service configuration to ensure GPU detection.
   • Optional: Restrict networking exposure with ufw or similar tools.

6. Verify the setup.

   • Test the Ollama instance by loading a small model:
     bashCopy

     ollama pull llama-7b
     ollama chat llama-7b

steps

1. Prepare a Docker Compose file:

   • Create a new directory for your Open WebUI setup and populate it with the following docker-compose.yml:
     yamlCopy

     version: '3.8'
     
     services:
       openwebui:
         image: openwebui/backend:latest
         container_name: openwebui
         labels:
           - traefik.http.routers.webui.rule=Host(`openwebui.local`)
           - traefik.http.routers.webui.tls=true
         environment:
           - OLLAMA_BASE_URL=http://10.20.0.20:11434
         networks:
           - traefik-network
         volumes:
           - openwebui_data:/app/backend/data
     
     networks:
       traefik-network:
         external: true
     
     volumes:
       openwebui_data:

   Replace 10.20.0.20 with your actual Ollama container's static IP address.

2. Deploy Open WebUI:

   bashCopy

   docker compose up -d

3. Test connectivity between WebUI and Ollama.

   • Access the WebUI via its FQDN or static IP.
   • Configure chat interfaces and verify that models load from Ollama.

4. Secure access.

   • Protect Open WebUI using Traefik with authentication middleware (e.g., Authentik or basic auth).
   • Test HTTPS connections.

5. Customize settings:

   • Add LLM models or set up extension features using the WebUI's admin interface.

steps

1. Enable Web Search in Open WebUI.

   • Navigate to Settings → Web Search and select a search engine (e.g., DuckDuckGo).

2. Add external API tokens:

   • Configure APIs like OpenAI's ChatGPT in the WebUI settings.

3. Test combined workflows.

   • Integrate local models for offline tasks and cloud-based APIs for access to external information.