Configuration

Carrot can be configured using environment variables which can be different depending on the deploying approach.

Django Backend

Configuration Section

Key	Description
`FRONTEND_URL`*	The URL for Frontend service which `Backend` should connect to. This only needs to include scheme (e.g. `http://`), host (`my-fronted.com`, or in this example, `frontend` in order to connect with service in the same compose stack) and optionally port, e.g., `3000`.
`ALLOWED_HOSTS`*	A list of strings representing the host/domain names that this Django site can serve.
`DB_ENGINE`*	The database backend to use. Carrot uses `Postgres` database backend.
`DB_HOST` `DB_PORT` `DB_NAME` `DB_USER` `DB_PASSWORD`*	These settings (`port`, `host`, `name`, `user`, `password`) are required for PostgreSQL database connection.
`DEBUG`	A boolean that turns on/off debug mode.
`WORKERS_URL`*	The URL for `Workers` (Azure functions) service which `Backend` should connect to. Structure is similar to `FRONTEND_URL`.
`WORKERS_UPLOAD_NAME`* `WORKERS_RULES_EXPORT_NAME`*	Name of queues in Azurite that `Workers` send messages about ScanReport uploads and Mapping Rules exports, respectively.
`WORKERS_RULES_NAME`	Name of point where `Backend` triggers Workers’ jobs. Default to `RulesOrchestrator`
`WORKERS_RULES_KEY`*	The key to authorise the request sent to `Workers` from `Backend`.
`STORAGE_CONN_STRING`*	The key to connect Backend and Azure local storage.
`SECRET_KEY`*	A secret key for a particular Django installation. This is used to provide cryptographic signing, and should be set to a unique, unpredictable value. Real value is used in Production.
`SIGNING_KEY`*	A key required in JWT token generation process for Next Auth. Real value is used in Production.

Examples

Below is the setting example for Carrot Backend or api service which is one part of Compose Stack used for local development.

api:
  image: carrot-backend
  build:
    context: app
    dockerfile: api/Dockerfile
  ports:
    - 8000:8000
  environment:
    - FRONTEND_URL=http://frontend:3000
    - ALLOWED_HOSTS=['localhost', '127.0.0.1','api', 'workers']
    - DB_ENGINE=django.db.backends.postgresql
    - DB_HOST=db
    - DB_PORT=5432
    - DB_NAME=postgres
    - DB_USER=postgres
    - DB_PASSWORD=postgres
    - DEBUG=True
    - WORKERS_UPLOAD_NAME=uploadreports-local
    - SECRET_KEY=secret
    - WORKERS_URL=http://workers:80
    - WORKERS_RULES_NAME=RulesOrchestrator
    - WORKERS_RULES_KEY=rules_key
    - WORKERS_RULES_EXPORT_NAME=rules-exports-local
    - STORAGE_CONN_STRING=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite:10000/devstoreaccount1;QueueEndpoint=http://azurite:10001/devstoreaccount1;TableEndpoint=http://azurite:10002/devstoreaccount1;
    - SIGNING_KEY=secret
  volumes:
    - ./app/api:/api
  depends_on:
    omop-lite:
      condition: service_completed_successfully

This service is built based on the Dockerfile inside app/api/ folder and exposed on port 8000:8000. It will start after the omop-lite service ran and did its jobs successfully. When running, it also uses the mounted code in the api folder so the changes will be reflected without restarting the stack.

You can see, edit, add or change environment variables on Azure web app by using Advanced edit on the page.

Below is just an example of variables that need adding to the Azure environment. Apart from these, there are some necessary variables that Azure may add to the environment automatically.

[
  { "other variables...": "values..." },
  {
    "name": "ALLOWED_HOSTS",
    "value": "['backend-url', 'localhost']",
    "slotSetting": false
  },
  {
    "name": "DATA_UPLOAD_MAX_MEMORY_SIZE",
    "value": "<you can configure the max size of uploading files here, defaulted to 2621440 B>",
    "slotSetting": false
  },
  {
    "name": "DB_ENGINE",
    "value": "django.db.backends.postgresql",
    "slotSetting": false
  },
  {
    "name": "DB_HOST",
    "value": "<db-host>",
    "slotSetting": false
  },
  {
    "name": "DB_NAME",
    "value": "<db-name>",
    "slotSetting": false
  },
  {
    "name": "DB_PASSWORD",
    "value": "<db-pass> (should be in Key vault)",
    "slotSetting": false
  },
  {
    "name": "DB_PORT",
    "value": "5432",
    "slotSetting": false
  },
  {
    "name": "DB_USER",
    "value": "<db-user>",
    "slotSetting": false
  },
  {
    "name": "DEBUG",
    "value": "0",
    "slotSetting": false
  },
  {
    "name": "FRONTEND_URL",
    "value": "<frontend-url>",
    "slotSetting": false
  },
  {
    "name": "SECRET_KEY",
    "value": "<key> should be in Key vault",
    "slotSetting": false
  },
  {
    "name": "SIGNING_KEY",
    "value": "<sign-key> should be in Key vault",
    "slotSetting": false
  },
  {
    "name": "STORAGE_CONN_STRING",
    "value": "<conn-key> should be in Key vault",
    "slotSetting": false
  },
  {
    "name": "WORKERS_RULES_EXPORT_NAME",
    "value": "rules-files",
    "slotSetting": false
  },
  {
    "name": "WORKERS_RULES_KEY",
    "value": "<workers-key> should be in Key vault",
    "slotSetting": false
  },
  {
    "name": "WORKERS_UPLOAD_NAME",
    "value": "upload",
    "slotSetting": false
  },
  {
    "name": "WORKERS_URL",
    "value": "<workers-url>",
    "slotSetting": false
  }
]

NextJS Frontend

Configuration Section

Key	Description
`BACKEND_URL`*	The URL for Backend service which `frontend` should connect to. This only needs to include scheme (e.g. `http://`), host (`my-fronted.com`, or in this example, `api` in order to connect with service in the same compose stack) and optionally port, e.g., `8000`.
`BACKEND_ORIGIN`	A list of extra safe origin domains from which Server Actions can be invoked. Next.js compares the origin of a Server Action request with the host domain, ensuring they match to prevent CSRF attacks. Only required in local dev.
`NODE_ENV` `WATCHPACK_POLLING`	These variables are set to enable `hot reload` feature when developing NextJS app.
`NEXTAUTH_BACKEND_URL`*	The backend base endpoint URL where `Frontend` send auth requests to.
`NEXTAUTH_SECRET`*	Used to encrypt the NextAuth.js JWT, and to hash email verification tokens. Real value is used in Production.
`NEXTAUTH_URL`*	The base URL where login and logout processes point to.

Examples

Below is the setting example for Carrot’s Frontend or frontend service which is one part of Compose Stack used for local development.

frontend:
  image: carrot-frontend
  build:
    context: app/next-client-app
    dockerfile: Dockerfile
    target: dev
  command: npm run dev
  ports:
    - 3000:3000
  environment:
    - BACKEND_URL=http://api:8000
    - BACKEND_ORIGIN=localhost:8000
    - NEXTAUTH_URL=http://localhost:3000/
    - NEXTAUTH_SECRET=verycomplexsecretkey
    - NEXTAUTH_BACKEND_URL=http://api:8000/api/
    - NODE_ENV=development
    - WATCHPACK_POLLING=true
  volumes:
    - ./app/next-client-app:/app
    - /app/node_modules
    - /app/.next
  depends_on:
    - api

This service is built based on the Dockerfile inside app/next-client-app/ folder and exposed on port 3000:3000. It will start after the api service is successfully up. When running, it also uses the mounted code in the next-client-app folder so the changes will be reflected without restarting the stack.

You can see, edit, add or change environment variables on Azure web app by using Advanced edit on the page.

Below is just an example of variables that need adding to the Azure environment. Apart from these, there are some necessary variables that Azure may add to the environment automatically.

[
  { "other variables...": "values..." },
  {
    "name": "BACKEND_URL",
    "value": "<backend-url>",
    "slotSetting": false
  },
  {
    "name": "NEXTAUTH_BACKEND_URL",
    "value": "<backend-url>/api/",
    "slotSetting": false
  },
  {
    "name": "NEXTAUTH_SECRET",
    "value": "<auth-secret> (should be in Key vault)",
    "slotSetting": false
  },
  {
    "name": "NEXTAUTH_URL",
    "value": "<frontend-url>",
    "slotSetting": false
  }
]

Azure functions

Configuration Section

Key	Description
`AzureWebJobsSecretStorageType`	Specifies the repository or provider to use for key storage. Carrot’s workers uses this as `files` for workers auth keys are persisted on the file system. In other words, we ask Docker to look for secrets locally which created at step 2 in the dev. quickstart. Only required in local development.
`IsEncrypted`	This setting represents whether the values in local.settings.json are encrypted using a local machine key. Only required in local development.
`FUNCTIONS_WORKER_RUNTIME`*	The language or language stack of the worker runtime to load in the function app.
`DB_ENGINE`*	The database backend to use. Carrot’s workers uses `Postgres` database backend.
`DB_HOST` `DB_PORT` `DB_NAME` `DB_USER` `DB_PASSWORD`*	These settings (`port`, `host`, `name`, `user`, `password`) are required for PostgreSQL database connection.
`RULES_QUEUE_NAME` `RULES_FILE_QUEUE_NAME` `WORKERS_UPLOAD_NAME`*	Name of queues in Azurite that `Workers` send messages about Rules actions triggers, Mapping Rules exports and ScanReport uploads, respectively.
`WEBSITE_HOSTNAME`	The address that can be used to reach the function app from outside. Only required in local development.
`STORAGE_CONN_STRING`* `AzureWebJobsStorage`*	The keys to connect Workers and Azure storage.

Examples

Below is the setting example for Carrot’s Azure functions or workers service which is one part of Compose Stack used for local development.

workers:
  image: carrot-workers
  build:
    context: app
    dockerfile: workers/Dockerfile
  ports:
    - 8080:80
    - 7071:80
  environment:
    - AzureWebJobsSecretStorageType=files
    - IsEncrypted=false
    - AzureWebJobsStorage=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite:10000/devstoreaccount1;QueueEndpoint=http://azurite:10001/devstoreaccount1;TableEndpoint=http://azurite:10002/devstoreaccount1;
    - FUNCTIONS_WORKER_RUNTIME=python
    - STORAGE_CONN_STRING=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite:10000/devstoreaccount1;QueueEndpoint=http://azurite:10001/devstoreaccount1;TableEndpoint=http://azurite:10002/devstoreaccount1;
    - APP_URL=http://api:8000/
    - WORKERS_UPLOAD_NAME=uploadreports-local
    - RULES_QUEUE_NAME=rules-local
    - RULES_FILE_QUEUE_NAME=rules-exports-local
    - WEBSITE_HOSTNAME=localhost:7071
    - DB_ENGINE=django.db.backends.postgresql
    - DB_HOST=db
    - DB_PORT=5432
    - DB_NAME=postgres
    - DB_USER=postgres
    - DB_PASSWORD=postgres
  volumes:
    - ./app/workers:/home/site/wwwroot
    - ./app/shared:/shared
    - ./app/workers/Secrets:/azure-functions-host/Secrets/
  depends_on:
    - api
    - azurite

This service is built based on the Dockerfile inside app/workers/ folder and exposed on ports 8080:80 and 7071:80. It will start after the api and azurite services are successfully up. When running, it also created some volumes that support its functionalities and authentication.

You can see, edit, add or change environment variables on Azure web app by using Advanced edit on the page.

Below is just an example of variables that need adding to the Azure environment. Apart from these, there are some necessary variables that Azure may add to the environment automatically.

[
  { "other variables...": "values..." },
  {
    "name": "AzureWebJobsStorage",
    "value": "<conn-key> same with STORAGE_CONN_STRING",
    "slotSetting": false
  },
  {
    "name": "DB_ENGINE",
    "value": "django.db.backends.postgresql",
    "slotSetting": false
  },
  {
    "name": "DB_HOST",
    "value": "<db-host>",
    "slotSetting": false
  },
  {
    "name": "DB_NAME",
    "value": "<db-name>",
    "slotSetting": false
  },
  {
    "name": "DB_PASSWORD",
    "value": "<db-pass> (should be in Key vault)",
    "slotSetting": false
  },
  {
    "name": "DB_PORT",
    "value": "5432",
    "slotSetting": false
  },
  {
    "name": "DB_USER",
    "value": "<db-user>",
    "slotSetting": false
  },
  {
    "name": "FUNCTIONS_EXTENSION_VERSION",
    "value": "~4",
    "slotSetting": false
  },
  {
    "name": "FUNCTIONS_WORKER_RUNTIME",
    "value": "python",
    "slotSetting": false
  },
  {
    "name": "STORAGE_CONN_STRING",
    "value": "<conn-key> should be in Key vault",
    "slotSetting": false
  },
  {
    "name": "RULES_FILE_QUEUE_NAME",
    "value": "rules-files",
    "slotSetting": false
  },
  {
    "name": "RULES_QUEUE_NAME",
    "value": "rules",
    "slotSetting": false
  },
  {
    "name": "WORKERS_UPLOAD_NAME",
    "value": "upload",
    "slotSetting": false
  }
]

Other services

Database

Carrot uses PostgreSQL database.

The Carrot Database would require two data components at the beginning:

An omop schema with loaded vocabularies
Seeding data about OMOP tables and fields names

For local development, the former can be created by omop-lite package as the example below (step 1 in developer quickstart guide) and the latter can be done by step 4 in developer quickstart guide.

Azurite

Carrot uses Azurite as queues and blobs storage for Azure functions.

This storage requires pre-made Queues and Blob Containers with purposes below:

Queue for Rules actions triggers, e.g., `rules`
Queue for Mapping rules exports, e.g., `rules-exports`
Queue for Scan reports uploads, e.g., `uploadreports`

Note: Queues names should match with RULES_QUEUE_NAME RULES_FILE_QUEUE_NAME WORKERS_UPLOAD_NAME in Azure functions.

Examples

The example below runs a PostgreSQL database for Carrot on port 5432:5432. Additionally, it runs Azure local storage for Carrot’s workers on ports 10000:10000, 10001:10001, and 10002:10002.

The command and AZURITE_ACCOUNTS environment variable in this example make sure the connection between azurite and workers proper.

After db is up, this service runs a DDL script to create a omop schema in the db, then load the vocabs downloaded from Athena there.

When omop schema existed, omop-lite will closes automatically.

db:
  image: postgres:13
  restart: always
  ports:
    - 5432:5432
  environment:
    - POSTGRES_PASSWORD=postgres
 
omop-lite:
  image: ghcr.io/andyrae/omop-lite
  volumes:
    - ./vocabs:/vocabs
  depends_on:
    - db
  environment:
    - DB_PASSWORD=postgres
    - DB_NAME=postgres
 
azurite:
image: mcr.microsoft.com/azure-storage/azurite
restart: always
volumes:
  - ./app/azurite:/azurite
ports:
  - 10000:10000
  - 10001:10001
  - 10002:10002
command: azurite --blobHost azurite --queueHost azurite --tableHost azurite --location /data --debug /data/debug.log --loose --skipApiVersionCheck
hostname: azurite
environment:
  - AZURITE_ACCOUNTS=devstoreaccount1:Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==

Uploading a Scan Report file Architecture Diagram