Deployment

Frisky scales down to run on your laptop, or can scale up on any major deployment platform today.

Local Cluster

Use LocalCluster when one Python process should manage the whole cluster.

from frisky import LocalCluster

with LocalCluster(n_workers=4, processes=True) as cluster:
    with cluster.get_client() as client:
        def increment(x):
            return x + 1

        futures = client.map(increment, range(10))
        total = client.submit(sum, futures)
        print(total.result())

Manual Deployment

Start the scheduler with the frisky CLI:

frisky scheduler --address 0.0.0.0:8786

Start workers that connect to it:

frisky worker 127.0.0.1:8786 --nthreads 4 --memory-limit 4GB

Connect from Python:

from frisky import Client

with Client("127.0.0.1:8786") as client:
    print(client.submit(lambda: "hello").result())

Deploy on Dask Clusters

Frisky can hijack any existing Dask cluster.

from dask.distributed import LocalCluster
import frisky

with LocalCluster() as dask_cluster:
    with dask_cluster.get_client() as dask_client:
        client = frisky.hijack(dask_client)

Dask has deployment solutions for all major deployment technologies, including HPC, Kubernetes, Cloud, SSH, and more.

Coiled

Frisky is developed alongside Coiled, a for-profit Dask service that runs on cloud.

import coiled
import frisky

cluster = coiled.Cluster(
    n_workers=20,
    worker_memory="16 GiB",
    region="us-east-1",
    arm=True,
)
dask_client = cluster.get_client()
client = frisky.hijack(dask_client)

Frisky and Coiled are owned by the same person, so it's likely that they'll work well together.