Architecture

This document uses the C4 Model for visualising the architecture of grand-challenge.org. This covers System Context, Containers (Applications/Data Stores), and Components (Modules/Libraries).

Ubiquitous Language

Users

Many different users and roles are supported on grand-challenge.org.

Researcher

A Researcher is a user who wants to manage medical imaging data and reports, collect annotations on medical imaging data, create a challenge for the data science community to generate solutions to clinical problems, and objectively evaluate the submitted algorithms to challenges.

Data Scientist

A Data Scientist is a user who participates in a challenge, downloads the training medical imaging data & annotations for a challenge, uploads algorithms for a challenge, and makes those algorithms available for clinicians to execute.

Clinician

A Clinician is a user who uses a workstation to learn how to annotate or read medical imaging data, uses a workstation to make annotations to medical imaging data, and uploads new medical imaging data for execution by algorithms.

System Context

An overview of the grand-challenge.org system, its users, and its system dependencies.

@startuml
!includeurl https://raw.githubusercontent.com/RicardoNiepel/C4-PlantUML/release/1-0/C4_Context.puml

LAYOUT_TOP_DOWN
LAYOUT_WITH_LEGEND()

title System Context diagram for grand-challenge.org

Person(clinicians, "Clinician", "User who annotates data, or has new clinical data for algorithm validation.")
Person(challenge_orgs, "Researcher", "User who poses challenges and compares state of the art algorithms.")
Person(data_scientists, "Data Scientist", "User who develops machine learning algorithms.")

System(grand_challenge, "grand-challenge.org", "Allows users to gather data, run workstations, train clinicians, annotate data, pose a challenge, evaluate solutions and deploy algorithms.")

System_Ext(grand_challenge_bridge, "Grand Challenge Bridge", "Connects to hospital systems, de-itentifies data, uploads data for algorithm execution and returns algorithm results.")
System_Ext(oauth, "Authentication", "Authenticates users through external providers (Google, GitHub).")
System_Ext(monitoring, "Monitoring and Alerting", "Monitors the system and sends alerts (Sentry, Prometheus).")
System_Ext(cdn, "Content Delivery Network", "Stores and serves static data (S3, CloudFront).")
System_Ext(mail, "E-mail", "Sends emails to users (SES).")

Rel(challenge_orgs, grand_challenge, "Uses")
Rel(data_scientists, grand_challenge, "Uses")

Rel(clinicians, grand_challenge, "Uses")
Rel(clinicians, grand_challenge_bridge, "Sends data from PACS to")
Rel(grand_challenge_bridge, grand_challenge, "Uploads data to")

Rel(grand_challenge, oauth, "Authenticates users against")
Rel(grand_challenge, mail, "Sends emails through")

Rel_Neighbor(grand_challenge, cdn, "Sends static data to")
Rel_Neighbor(monitoring, grand_challenge, "Scrapes monitoring endpoints of")
@enduml

Containers

The overall shape of the architecture and technology choices. Note: a container is a separately deployable application or data store, not necessarily a Docker container.

@startuml
!includeurl https://raw.githubusercontent.com/RicardoNiepel/C4-PlantUML/release/1-0/C4_Container.puml

LAYOUT_TOP_DOWN
LAYOUT_WITH_LEGEND()

title Container diagram for grand-challenge.org

Person(clinicians, "Clinician", "User who annotates data, or has new clinical data for algorithm validation.")
Person(challenge_orgs, "Researcher", "User who poses challenges and compares state of the art algorithms.")
Person(data_scientists, "Data Scientist", "User who develops machine learning algorithms.")

System_Boundary(s1, "grand-challenge.org") {
    Container(gc_web_app, "API", "Nginx, Django", "Provides the grand challenge functionality via HTML and a REST API.")
    Container(gc_algorithm_worker, "Algorithm and Evaluation Worker", "Django, Docker, Celery, Redis", "Executes algorithm and evaluation container images on medical imaging data.")
    Container(gc_workstation_worker, "Workstation Worker", "Nginx, Django, Docker, Traefik, Celery, Redis", "Instantiates and proxies workstation container images for users.")

    Container(workstation_instance, "Web Workstation", "SPA, MeVisLab, VueJS, Docker", "Provides a web based workstation for server side rendering of medical imaging data. 1 container image per active user.")

    ContainerDb(postgres, "Database", "RDS/PostgreSQL", "Stores user information, medical imaging metadata, job results etc.")

    Container(protected_s3, "Protected Data Store", "S3/Minio", "Stores protected medical imaging objects.")
    Container(private_s3, "Private Data Store", "S3/Minio", "Stores private objects, container images etc.")
}

System_Ext(oauth, "Authentication", "Authenticates users through external providers (Google, GitHub).")
System_Ext(monitoring, "Monitoring and Alerting", "Monitors the system and sends alerts (Sentry, Prometheus).")
System_Ext(cdn, "Content Delivery Network", "Stores and serves static data (S3, CloudFront).")
System_Ext(mail, "E-mail", "Sends emails to users (SES).")

Rel(challenge_orgs, gc_web_app, "Uses", "HTTPS, HTML, JSON")
Rel(data_scientists, gc_web_app, "Uses", "HTTPS, HTML, JSON")
Rel(clinicians, gc_web_app, "Uses", "HTTPS, HTML")

Rel(gc_web_app, oauth, "Authenticates users against", "HTTPS, OAuth2")
Rel(s1, mail, "Sends emails through", "SMTP")
Rel(gc_web_app, cdn, "Sends static data to", "HTTPS")
Rel(monitoring, s1, "Scrapes monitoring endpoints of", "HTTPS")

Rel(gc_web_app, postgres, "Reads and writes to", "SSL, SQL")
Rel(gc_web_app, protected_s3, "Stores objects in, and proxies access to", "HTTPS, S3")
Rel(gc_web_app, private_s3, "Stores objects in", "HTTPS, S3")

Rel(gc_workstation_worker, postgres, "Reads and writes to", "SSL, SQL")
Rel(gc_workstation_worker, private_s3, "Fetches container images from", "HTTPS, S3")
Rel_Neighbor(gc_workstation_worker, workstation_instance, "Instantiates and proxies", "HTTPS, Docker")
Rel_U(gc_workstation_worker, clinicians, "Provides SPA workstation to", "HTTPS, WSS, HTML, JSON")

Rel_U(gc_algorithm_worker, postgres, "Reads and writes to", "SSL, SQL")
Rel_U(gc_algorithm_worker, private_s3, "Fetches container images from", "HTTPS, S3")
Rel_U(gc_algorithm_worker, protected_s3, "Reads objects from", "HTTPS, S3")
@enduml

Components

Logical modules and their interactions with containers. Note: a component is a grouping of related functionality behind a well defined interface, not components in the sense of pipelines. Grand-challenge.org is a Django monolith, but several components can be deployed and scaled independently.

API

The API layer provides the HTML views and REST services to the user.

@startuml
!includeurl https://raw.githubusercontent.com/RicardoNiepel/C4-PlantUML/release/1-0/C4_Component.puml

LAYOUT_TOP_DOWN
LAYOUT_WITH_LEGEND()

title Component diagram for grand-challenge.org - API

Person(users, "Users", "")

Container_Boundary(s1, "API") {
    Component(gc_web_app, "Web Application", "Django", "Provides the grand challenge functionality via HTML and a REST API.")
    Component(nginx, "Reverse Proxy", "Nginx", "SSL termination and routing")
    Component(memcached, "Cache", "Memcached", "Local in-memory cache of expensive data.")
}

ContainerDb(postgres, "Database", "RDS/PostgreSQL", "Stores user information, medical imaging metadata, job results etc.")
Container(message_queue, "Message Queue", "SQS/Redis", "Queue of units of work Celery messages.")
Container(protected_s3, "Protected Data Store", "S3/Minio", "Stores protected medical imaging objects.")
Container(private_s3, "Private Data Store", "S3/Minio", "Stores private objects, container images etc.")

System_Ext(oauth, "Authentication", "Authenticates users through external providers (Google, GitHub).")
System_Ext(cdn, "Content Delivery Network", "Stores and serves static data (S3, CloudFront).")

Rel(users, nginx, "Use", "HTTPS, HTML, JSON")

Rel(gc_web_app, oauth, "Authenticates users against", "HTTPS, OAuth2")
Rel(gc_web_app, cdn, "Sends static data to", "HTTPS")

Rel(gc_web_app, postgres, "Reads and writes to", "SSL, SQL")
Rel(gc_web_app, protected_s3, "Stores objects in, and proxies access to", "HTTPS, S3")
Rel(gc_web_app, private_s3, "Stores objects in", "HTTPS, S3")
Rel(gc_web_app, message_queue, "Puts messages on", "SSL, SQS/Redis, Celery")
Rel_Neighbor(gc_web_app, memcached, "Uses", "HTTP")

Rel(nginx, gc_web_app, "Proxies", "HTTP, HTML, JSON")

@enduml

Workstations

Grand Challenge is able to launch workstations for users. Workstations are container images that allow for interaction with medical data in the browser. One workstation container image is instantiated per user, which provides a single page application. A secure web socket connection is then established between the user and their container instance.

@startuml
!includeurl https://raw.githubusercontent.com/RicardoNiepel/C4-PlantUML/release/1-0/C4_Component.puml

LAYOUT_WITH_LEGEND()

title Component diagram for grand-challenge.org - Workstation Worker

Person(workstation_user, "Workstation User", "User who wants to view a workstation.")

ContainerDb(postgres, "Database", "RDS/PostgreSQL", "Stores user information, medical imaging metadata, job results etc.")
Container(private_s3, "Private Data Store", "S3/Minio", "Stores private objects, container images etc.")
Container(message_queue, "Message Queue", "SQS/Redis", "Queue of units of work Celery messages.")
Container(gc_web_app, "API", "Nginx, Django", "Provides the grand challenge functionality via HTML and a REST API.")

Container_Boundary(api, "Workstation Worker") {
    Component(nginx, "Reverse Proxy", "Nginx", "SSL termination and routing")
    Component(webapp, "Web Application", "Django, Celery", "Authorization for workstation connections, manages workstation container lifecycles.")
    Component(traefik, "Workstation Reverse Proxy", "Traefik", "Routes users to their workstation service")
    Component(docker, "Compute Pool", "Docker", "Provides compute resources for workstation services, 1 workstation container instance per user.")

    Rel(nginx, webapp, "Fetches routing information from", "HTTP")
    Rel(nginx, traefik, "Internally redirects to", "HTTP, WS")

    Rel(traefik, docker, "Proxies", "HTTP, WS")
    Rel(webapp, docker, "Schedules workstation services on", "HTTPS, JSON")
}

Rel(workstation_user, nginx, "Uses", "HTTPS, WSS, JSON")

Rel(webapp, message_queue, "Fetches messages from", "SSL, SQS/Redis, Celery")
Rel(webapp, postgres, "Reads and writes to", "SSL, SQL")
Rel(webapp, private_s3, "Fetches container images from", "HTTPS, S3")

Rel(docker, gc_web_app, "Fetches medical images, algorithm results, etc, from", "HTTPS, JSON")

@enduml

Image, Evaluation and Algorithm Workers

There are several tasks that require asynchronous processing that require long compute times, high memory usage and potentially the docker api and/or a GPU. These tasks include medical image importing, challenge submission evaluation (requires docker), and algorithm execution (requires docker and a GPU). These tasks are scheduled using Celery, and then executed by worker nodes that can horizontally scale.

@startuml
!includeurl https://raw.githubusercontent.com/RicardoNiepel/C4-PlantUML/release/1-0/C4_Component.puml

LAYOUT_TOP_DOWN
LAYOUT_WITH_LEGEND()

title Component diagram for grand-challenge.org - Images, Evaluation and Algorithm Workers

Container_Boundary(s1, "Celery Worker") {
    Component(gc_algorithm_worker, "Web Application", "Django, Celery", "Manages container lifecycle and generates job results.")
    Component(docker, "Compute Pool", "Docker", "Provides compute resources for container images.")
}

ContainerDb(postgres, "Database", "RDS/PostgreSQL", "Stores user information, medical imaging metadata, job results etc.")
Container(protected_s3, "Protected Data Store", "S3/Minio", "Stores protected medical imaging objects.")
Container(private_s3, "Private Data Store", "S3/Minio", "Stores private objects, container images etc.")
Container(message_queue, "Message Queue", "SQS/Redis", "Queue of units of work Celery messages.")

Rel(gc_algorithm_worker, message_queue, "Fetches messages from", "SSL, SQS/Redis, Celery")

Rel(gc_algorithm_worker, postgres, "Reads and writes to", "SSL, SQL")
Rel(gc_algorithm_worker, private_s3, "Fetches container images from", "HTTPS, S3")
Rel(gc_algorithm_worker, protected_s3, "Reads objects from", "HTTPS, S3")

Rel(gc_algorithm_worker, docker, "Provisions volumes, instantiates containers with GPU access on", "HTTPS, JSON")

@enduml