Skip to content

GPU passthrough

1- Install nvidia containter toolkit

Follow the instructions here for your operating system: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

2- Change the smartscope config to use the GPU

Edit smartscope.yml to use the smartscopeai gpu image and configure the passthrough: 

  smartscope_ai:
    user: ${UID}:${GID}
    image: ghcr.io/smartscope-org/smartscopeai:dev-gpu
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            device_ids: ['0'] ##YOU CAN CHANGE THE ID
            capabilities: [gpu]

3- Restart SmartScope

./smartscope.sh restart

Installing SmartScopeAI without containerization

Requirements

Install the uv python package manager. With or without sudo:

curl -LsSf https://astral.sh/uv/install.sh | sh Or, wget -qO- https://astral.sh/uv/install.sh | sh

Setup your Docker environment to allow SmartScopeAI connection

1- Add a password on the cache

First, protect the redis server in the cache container with a password since it will now be exposed to the outside network.

In the smartscope installation directory create a redis.conf file: 

bind 0.0.0.0
requirepass YOURREDISPASSWORD
save ""
appendonly no

2- Change the cache service

In smartscope.yml, change the following to the cache service to expose it ouside of the docker network

services:
  [...]
  cache:
    volumes:
      - ./redis.conf:/redis.conf
    ports:
      - 59000:6379
    command: redis-server /redis.conf

Your cache service will now be protetected by the password and exposed to the host on port 59000

3- Change smartscope.conf to add the gpu queue and the cache password

REDIS_PASSWORD=redisPassword
QUEUES=gpu,smartscope_ai_cpu
TRANSIENT_QUEUES=gpu

This will set the priority to the gpu worker and fall back to the smartscope_ai service (boud tot the smartscop_ai_cpu queue) if unavailable or busy.

Download and install SmartScopeAI

git clone https://github.com/smartscope-org/SmartScopeAI.git -b dev cd SmartScopeAI uv sync --extra cuda

Dowload the model weights (TEMPLATE_FILES)

COMING SOON

Set up the environment

Create a .env file and add the following:

REDIS_HOST=xxx.xxx.xxx.xxx
REDIS_PORT=59000
REDIS_PASSWORD=YOURREDISPASSWORD
CALLBACK_SITE=http:/localhost:48000
TEMPLATE_FILES=/PATH/TO/TEMPLATE_FILES/
WORKER_QUEUES=gpu
SCRATCH_DIR=/PATH/TO/SCRATCH/

Start the worker

uv run --env-file .env celery -A SmartscopeAI.interfaces.celery.app worker --loglevel=DEBUG --pool=solo