Working with Local LLMs on the Linux Command Line

Overview: One Local Runtime, Several Workflows

The common component is Ollama. It stores and runs local models and exposes an HTTP API on localhost. A model can be used directly from the shell with ollama run, called programmatically through the API, or exposed to a client such as OpenCode through its OpenAI-compatible endpoint.

There are three practical routes. A smartphone scanner can capture, deskew and crop the document close to the source. Classical Tesseract is fast and deterministic. GLM-OCR uses a vision-capable local model and can be preferable when the visual structure of the document matters. A useful workflow is therefore to capture a clean image first, attempt local GLM-OCR, and retain Tesseract as a robust fallback.

Core design of myollamaocr

The script does not pass a potentially huge Base64 image as a shell argument. It writes the encoded image to a temporary file and lets jq --rawfile read it into the JSON request. This avoids shell argument-size limits. The request explicitly caps both output size and total HTTP runtime to prevent runaway generations.

MODEL="glm-ocr"
OLLAMA_URL="http://127.0.0.1:11434/api/generate"

MAX_TOKENS=4096
MAX_TIME=120

base64 -w 0 "$INPUT" > "$IMAGE64"

jq -n \
    --rawfile image "$IMAGE64" \
    --arg model "$MODEL" \
    --argjson max_tokens "$MAX_TOKENS" \
    '{
        model: $model,
        prompt: "Text Recognition:",
        images: [$image],
        stream: false,
        options: {
            num_predict: $max_tokens,
            temperature: 0.0,
            top_k: 1,
            top_p: 0.00001,
            repeat_penalty: 1.1
        }
    }' > "$REQUEST"

curl \
    --silent \
    --show-error \
    --max-time "$MAX_TIME" \
    --output "$RESPONSE" \
    --write-out '%{http_code}' \
    --header 'Content-Type: application/json' \
    --data-binary "@$REQUEST" \
    "$OLLAMA_URL"

jq -er '.response' "$RESPONSE" > "$RESULT"
mv -- "$RESULT" "$OUTPUT"

The model prompt is intentionally minimal: Text Recognition:. The image is sent to Ollama's /api/generate endpoint, generation is non-streaming, and conservative sampling settings are used because OCR should reproduce rather than invent text. The final .response field is extracted with jq and written to a text file with the same basename as the image.

Batch selection with myocrbatch

The wrapper has two deliberately simple modes. By default it delegates to myollamaocrbatch, whose intended behavior is “GLM-OCR first, Tesseract as fallback”. With -t, Ollama is bypassed completely and every JPG/PNG file is processed by Tesseract only.

# Default mode: delegate to the GLM-OCR batch wrapper
if [ "$TESSERACT_ONLY" -eq 0 ]; then
    exec myollamaocrbatch
fi

# -t mode: Tesseract only
for file in "${files[@]}"; do
    target="${file%.*}_tess.txt"

    tesseract "$file" stdout -l deu --psm 4 --oem 1 \
        -c preserve_interword_spaces=1 > "$target"
done

The central Tesseract command

tesseract "$file" stdout -l deu --psm 4 --oem 1 \
    -c preserve_interword_spaces=1 > ${file%.*}_tess.txt

Connecting Ollama Cloud to OpenCode

1. Sign in to an Ollama account, create an API key under Settings → Keys, then start OpenCode.
2. In OpenCode run /connect, select Ollama Cloud, and enter the Ollama Cloud API key.
3. Before the cloud model is selected in OpenCode, pull its model information with the local Ollama command. For example:

   ollama pull gpt-oss:20b-cloud

   For a cloud model this step prepares the local Ollama-side model information; the actual model inference remains remote rather than requiring the complete model weights to run on the local GPU or CPU.
4. Run /models in OpenCode and select the desired Ollama Cloud model.

Connecting OpenRouter to OpenCode

1. Create an OpenRouter account, add credits if paid models are required, and create an API key.
2. Run /connect in OpenCode and select OpenRouter.
3. Enter the OpenRouter API key.
4. Run /models. Many OpenRouter models are already known to OpenCode and can be selected immediately. Free variants can be used for experiments; paid variants draw from the OpenRouter credit balance.

If only selected OpenRouter models should appear in the OpenCode configuration, they can be declared explicitly in opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "openrouter": {
      "models": {
        "z-ai/glm-5.2": {},
        "z-ai/glm-5.2:free": {}
      }
    }
  }
}

The exact model IDs must match the IDs exposed by OpenRouter. Authentication does not need to be stored in this JSON file when it has already been added with /connect.

A coding model can be queried directly with Ollama, for example:

ollama run qwen2.5-coder:7b "Explain this Bash function and simplify it."

For actual software work, OpenCode provides the more useful abstraction. It can operate inside a project, inspect files, apply edits and invoke shell commands. The supplied configuration exposes several Ollama models to OpenCode through the local OpenAI-compatible API.

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://127.0.0.1:11434/v1"
      },
      "models": {
        "qwen3:8b": { "name": "Qwen 3 8B" },
        "qwen2.5-coder:7b": { "name": "Qwen 2.5-coder 7B" },
        "qwen3.5:4b": { "name": "Qwen 3.5 4B" },
        "glm-ocr:latest": {
          "name": "GLM-OCR",
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          }
        },
        "granite4.1:3b": { "name": "Granite 4.1 3B" }
      }
    }
  }
}

YouTube automatically generated subtitles are a convenient source for lectures, talks and long technical videos, but raw VTT files contain timestamps, metadata, markup, blank lines and frequently duplicated auto-caption lines. The supplied myyoutubesubs script separates acquisition from cleanup.

# Download automatically generated subtitles
yt-dlp --write-auto-subs --skip-download "$URL"

# Keep only VTT files created by this invocation
mapfile -t VTTFILES < <(
    find . -maxdepth 1 -type f -name '*.vtt' -newer "$MARKER" -printf '%T@ %p\n' |
    sort -n |
    cut -d' ' -f2-
)

# Remove VTT metadata/timestamps/empty lines, strip HTML-like tags,
# de-duplicate consecutive identical subtitle lines, join to flowing text
# and wrap at the configured line length.
awk '
    /^WEBVTT/                   { next }
    /^Kind:/                    { next }
    /^Language:/                { next }
    /^NOTE/                     { next }
    /-->/                       { next }
    /^[[:space:]]*$/            { next }

    {
        gsub(/<[^>]*>/, "")
        sub(/^[[:space:]]+/, "")
        sub(/[[:space:]]+$/, "")

        if (length($0) && $0 != previous) {
            print
            previous = $0
        }
    }
' "$dst_vtt" |
tr '\n' ' ' |
sed 's/[[:space:]]\+/ /g; s/^ //; s/ $//' |
fold -s -w "$LINELENGTH" > "$dst_txt"

For a single video the script produces ausgabe.vtt and cleaned ausgabe.txt. For playlists it generates numbered pairs such as ausgabe_001.vtt and ausgabe_001.txt. Option -a additionally downloads the best available audio stream, while -l controls the output line width.

Pass prompt and source text together with cat

A particularly shell-friendly pattern is to keep the instruction in one file and the transcript in another, concatenate both, and pass standard input to the model:

cat summary_prompt.txt ausgabe.txt |
    ollama run qwen3:8b > condensed.txt

A prompt file could contain only the instruction, for example:

Summarize the following transcript in concise English.
Preserve the essential technical arguments, commands and conclusions.
Do not invent information that is not present in the source.

--- TRANSCRIPT ---

The same principle works without a separate prompt file. A shell group can prepend the instruction and then stream the source text:

{
    printf '%s\n\n' \
      'Summarize the following text concisely while preserving technical details:'
    cat ausgabe.txt
} | ollama run qwen3:8b > condensed.txt

[mcp_servers.example]
url = "https://www.drbreinlinger.de/mcp/example"

[mcp_servers.example.env_http_headers]
Authorization = "example_MCP_AUTHORIZATION"

[mcp_servers.example.tools.get_database_semantics]
approval_mode = "approve"

[mcp_servers.example.tools.query_sql]
approval_mode = "approve"

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://127.0.0.1:11434/v1"
      },
      "models": {
        "qwen3:8b": {
          "name": "Qwen 3 8B"
        },
        "qwen2.5-coder:7b": {
          "name": "Qwen 2.5-coder 7B"
        },
        "qwen3.5:4b": {
          "name": "Qwen 3.5 4B"
        },
        "glm-ocr:latest": {
          "name": "GLM-OCR",
          "modalities": {
            "input": [
              "text",
              "image"
            ],
            "output": [
              "text"
            ]
          }
        },
        "granite4.1:3b": {
          "name": "Granite 4.1 3B"
        }
      }
    }
  },
  "mcp": {
    "example": {
      "type": "remote",
      "url": "https://www.drbreinlinger.de/mcp/example",
      "oauth": false,
      "headers": {
        "Authorization": "Bearer {env:EXAMPLE_MCP_TOKEN}"
      },
      "enabled": true
    },
  "model": "granite4.1:3b"
}

{
  "mcpServers": {
    "example": {
      "type": "http",
      "url": "https://www.drbreinlinger.de/mcp/example",
      "headers": {
        "Authorization": "Bearer ${EXAMPLE_MCP_TOKEN}"
      }
    }
  }
}

Environment variables for credentials

export EXAMPLE_MCP_TOKEN='...'
export example_MCP_AUTHORIZATION='Bearer ...'

The two configuration formats expect slightly different values: the OpenCode and Claude examples construct the Bearer prefix inside the configuration, whereas the supplied Codex TOML maps the environment variable directly to the complete Authorization header.

Direct Comparison of the Main Workflows

Official References

• Ollama Linux installation and API documentation: docs.ollama.com
• OpenCode installation, models, providers, Ollama Cloud, OpenRouter, configuration and MCP documentation: opencode.ai/docs
• Ollama Cloud documentation and account management: ollama.com / docs.ollama.com
• OpenRouter API, models and credits: openrouter.ai
• Codex MCP configuration documentation: developers.openai.com/codex/mcp
• Claude Code MCP documentation: docs.anthropic.com/en/docs/claude-code/mcp