Personal ChatGPT with Ollama & Open-WebUI

Prerequisites

• A Computer with the appropriate specifications.
  I am running mine as an Ubuntu VM in a ProxMox environment with the following specifications:
  • 4 vCPU
  • 16 GB RAM
  • 200GB Storage (May need more for storing large models)
  • RTX 2080Ti 11GB passed through directly to the VM
• Docker installed, if you arent familiar with this see 'Install Docker'
• Comfortable with Ubuntu terminal.

Install Updates and Ubuntu Nvidia Drivers

This is my twist on this Canonical blog post here: https://ubuntu.com/blog/deploying-open-language-models-on-ubuntu

If you're running something other than Ubuntu you will need to adapt these commands to fit your operating system.
First we'll be doing some basic Ubuntu updating to make sure everything is up-to-date and that you have drivers installed for your GPU.
To do this run the following commands.

sudo apt update -y
sudo apt full-upgrade -y
sudo apt install -y ubuntu-drivers-common
sudo ubuntu-drivers install
sudo systemctl reboot

Install & Run Ollama using official script

Source: https://ollama.com/download/linux

Next, we need to pull the install script for Ollama

curl -fsSL https://ollama.com/install.sh | sh

Afterwards, pick a model and run it.
The first time will take a while as it needs to download the model, Official Models are listed here: https://ollama.com/library/llama3
In my case I chose llama3:8b model

Download & Run 'Open-WebUI'

ollama run llama3:8b

This will make our AI a little friendlier to use by giving us a web interface to visit and use on our network.
If you'd prefer the official instructions are also available here: https://docs.openwebui.com/getting-started/
For this we will be using Docker to make things easy and quick to deploy, ensure you have Docker setup as listed in the the prerequisites!

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

You can validate the container is up and running with a quick docker ps command

Explore

Open a web browser on any device in your network (assuming you've not enabled any firewall policies on your Ubuntu machine) and navigate to your Ubuntu machines IP address and port 8080 to log into OpenWebUI eg. http://192.168.0.10:8080
From here register to create your account. The first account registered which will automatically be added with Administrator privileges.

If you'd prefer to interact with your Ollama instance via it's API or have another application or service that hooks in, the default Ollama port is 11434.

Extra's

GPU Monitoring

To watch GPU Utilization in real time run the following command then ask something of your AI instance.

sudo watch -n2 nvidia-smi

Additional App Support

If you plan on using this with other applications you may also needed to adjust the startup variables for Ollama to support additional origins.
To do so run the following:

sudo systemctl edit ollama.service

If prompted select your preferred editor, mine's Nano.

Add or Adjust the following lines as needed, in my case I needed to add support of Obsidian to enable AI Plugins for my documentation.

[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_ORIGINS=app://obsidian.md*"