DeskLLM - Guide

Getting Started

1. Download a Model

DeskLLM needs a language model to work. On first launch, click the download icon in the bottom toolbar to open the Model Manager.

Recommend tab — Choose a model that fits your computer:

Click Download on any model card. The download progress is shown in real time.

Browse tab — Search Hugging Face for any GGUF model by keyword.

2. Start Chatting

1. Select a model from the dropdown in the bottom toolbar
2. Click the load icon (the model loads automatically on first message if not loaded)
3. Type your message and press Return to send
4. The AI responds in real time, token by token

Interface Overview

Sidebar

Toggle with the hamburger icon (top-left). Shows your chat sessions.

• Click a session to switch to it
• Right-click to Rename or Delete
• Click + New Chat to start a fresh conversation

Chat Area

• Your messages appear on the right (colored bubbles)
• AI responses appear on the left
• Click the copy icon on any message to copy it to clipboard
• Press Shift + Return for a new line without sending

Bottom Toolbar

Model Management

Open with the download icon in the toolbar.

• Downloaded tab — Lists all models on your disk. Delete models you no longer need to free space.
• Recommend tab — Curated list of popular models with one-click download. Disk space is shown before downloading.
• Browse tab — Search Hugging Face directly. Filter by name and browse available GGUF files for any repository.

Where Models Are Stored

Settings

Access from the gear icon (bottom-left) or the app menu.

Choosing the Right Settings

• Temperature — Lower (0.1–0.3) for factual answers. Higher (0.7–1.0) for creative writing.
• Max Tokens — Increase for longer responses. 512 is good for most conversations.
• Context Size — Larger values let the model remember more of the conversation, but use more RAM.

REST API

DeskLLM runs a local API server compatible with the OpenAI format. Other apps on your computer can use it.

Endpoint: http://localhost:8800

Example (curl)

curl http://localhost:8800/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

Example (Python)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8800/v1", api_key="none")
response = client.chat.completions.create(
    model="local",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Keyboard Shortcuts

Tips

• Switch models freely — You can change models mid-conversation at any time.
• System prompt — Customize it for different use cases: translator, coding assistant, writing helper, etc.
• Multiple sessions — Keep separate conversations for different topics. The sidebar makes it easy to switch.
• Offline use — Once a model is downloaded, DeskLLM works completely offline.
• API integration — Use the built-in API to connect DeskLLM with other tools, scripts, or apps on your machine.

Troubleshooting

Privacy

• All AI processing happens locally on your computer
• No data is sent to any external server
• Chat history is stored only on your local disk
• Models are downloaded from Hugging Face (the only network access)