Here is a quick step by step.
Install Ubuntu 24.04 Desktop. At the time Ubuntu Server 24.04 has issues. So run this on Desktop
Login and open a terminal
sudo su –
apt-get update
apt-get upgrade
nvidia-smi
(this should show you your NVidia GPU. These drivers work well. Keep them)
apt install curl
curl -fsSL https://ollama.com/install.sh | sh
If it errors, run it a second time. It fixes the error.
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama2
ollama run llama2
Open a new terminal
watch -n 0.5 nvidia-smi
You can see the GPU work
Back in the LLama2 terminal
ctrl +c
this will kill the session
Add Docker’s official GPG key by selecting below and pasting it into the terminal:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo \
“deb [arch=$(dpkg –print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo “$VERSION_CODENAME”) stable” | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
After the key installs:
apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
docker run -d –network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 –name open-webui –restart always ghcr.io/open-webui/open-webui:main
sudo docker ps
Now open a browser and go to localhost:8080
Signup and create an account. this will be the administrator
Back in the Terminal, setup other modes:
ollama pull codegemma
ollama pull llama2:70b
ollama pull llama3:70b
ollama pull llama3
ollama pull dolphin-llama3:8b
ollama pull dolphin-llama3:8b-256k
ollama pull dolphin-llama3:70b-256k
ollama pull dolphin-llama3:70b-256k
ollama pull dolphin-llama3:70b
ollama pull llava
Now go and have fun
