Storage

The Storage module handles file and image uploads with automatic validation, image compression and resizing, thumbnail generation, and multiple backend providers.

Install:

pip install "kirak[storage]"

Enable:

app = create_kirak_app(models_path="...", modules=["storage"])

Providers

kirak.json lists the storage provider instances the app can use and names the default. Each instance has a name (the key) and a type (which implementation to use). Several instances can be active at once, including two of the same type, for example two S3 buckets.

{
  "storage": {
    "default_provider": "main",
    "providers": {
      "main":    { "type": "aws", "aws_bucket": "app-assets", "aws_region": "us-east-1" },
      "archive": { "type": "aws", "aws_bucket": "app-archive" },
      "local":   { "type": "local", "upload_dir": "assets/media" }
    }
  }
}

Built-in `type`	Backend	Settings in the instance entry
`local`	Files saved to disk, served by `StaticFiles` at `/media/<instance name>`	`upload_dir` (default `assets/media`)
`aws`	AWS S3	`aws_bucket` (required), `aws_region`, `endpoint_url`, `public_url`, `acl`
`wasabi`	Wasabi (S3-compatible)	same as `aws`; `endpoint_url` defaults to `https://s3.wasabisys.com`
`r2`	Cloudflare R2 (S3-compatible)	`account_id` (required), `aws_bucket` (required), `jurisdiction` (`default`, `eu`, `fedramp`), `public_url`
`spaces`	DigitalOcean Spaces (S3-compatible)	`aws_bucket` (required), `aws_region` (required, e.g. `fra1`), `public_url`, `acl` (default `public-read`)
`cubbit`	Cubbit DS3 (S3-compatible, EU)	same as `aws`; `endpoint_url` defaults to `https://s3.cubbit.eu`, `aws_region` to `eu-west-1`
`ovh`	OVHcloud Object Storage (S3-compatible)	`aws_bucket` (required), `aws_region` (required, e.g. `gra`, `de`, `eu-west-par`), `public_url`, `acl`
`b2`	Backblaze B2 (S3-compatible)	`aws_bucket` (required), `aws_region` (required, e.g. `us-west-004`), `public_url`
`gcs`	Google Cloud Storage (`pip install "kirak[storage-gcs]"`)	`bucket` (required), `project_id`, `public_url`, `signing_service_account`
`azure`	Azure Blob Storage (`pip install "kirak[storage-azure]"`)	`container` (required), `account_name` (required without a connection string), `account_url`, `public_url`

The full list of settings and secrets per type is in Configuration, and kirak providers storage <type> prints it with a kirak.json example.

Cloudflare R2. The endpoint is built from account_id (and jurisdiction for EU or FedRAMP buckets). R2’s S3 endpoint is never publicly readable, so set public_url to the bucket’s r2.dev subdomain or a custom domain bound to the bucket; without it, use get_url with expires. R2 has no object ACLs, so acl must stay unset. Create the keys as an R2 API token (access key id and secret).

DigitalOcean Spaces. aws_region is the Space’s datacenter and picks the endpoint. Spaces files are private unless uploaded with a public ACL, so uploads are sent with public-read unless acl is private. Returned URLs use https://<bucket>.<region>.digitaloceanspaces.com; set public_url to the CDN endpoint (https://<bucket>.<region>.cdn.digitaloceanspaces.com) or your custom domain to serve through the CDN.

Cubbit DS3. A custom tenant sets endpoint_url to https://s3.<tenant>.cubbit.eu.

OVHcloud Object Storage. aws_region is the region code in lower case (gra, sbg, rbx, de, uk, waw, bhs, eu-west-par, …) and picks the https://s3.<region>.io.cloud.ovh.net endpoint. Files are private unless a bucket policy makes them public, or acl is public-read. Returned URLs use https://<bucket>.s3.<region>.io.cloud.ovh.net; set public_url for a CDN or custom domain. The keys are an S3 user’s access key and secret key from the OVHcloud Control Panel.

Backblaze B2. aws_region is the region in the bucket’s S3 endpoint (s3.us-west-004.backblazeb2.com -> us-west-004). B2 sets ACLs per bucket, and refuses a different ACL on a file, so acl must stay unset: make the bucket public or private in Backblaze. Returned URLs are https://s3.<region>.backblazeb2.com/<bucket>/<path>, readable when the bucket is public; set public_url for a CDN or the bucket’s friendly URL (https://f004.backblazeb2.com/file/<bucket>). The keys are an application key: its keyID as the access key and the applicationKey as the secret key.

Google Cloud Storage. Credentials come from KIRAK_STORAGE_<INSTANCE>_CREDENTIALS_JSON (the contents of a service account key file) or, when it is unset, from Application Default Credentials (the service account of Cloud Run, GKE or a VM, or gcloud auth application-default login). Without either, the instance fails with a configuration error when it is first used. Returned URLs are https://storage.googleapis.com/<bucket>/<path>, readable only if the bucket grants allUsers the Storage Object Viewer role; set public_url for a Cloud CDN or load balancer domain. get_url with expires returns a V4 signed URL: with a key it is signed locally; under Application Default Credentials there is no private key, so set signing_service_account to a service account the app’s identity may sign as (iam.serviceAccounts.signBlob, e.g. the Service Account Token Creator role on it). Without it, get_url with expires raises a configuration error. Deleting a file that no longer exists succeeds, as on S3.

Azure Blob Storage. Credentials, first match wins: KIRAK_STORAGE_<INSTANCE>_CONNECTION_STRING; account_name with KIRAK_STORAGE_<INSTANCE>_ACCOUNT_KEY; account_name alone, using the app’s Azure identity (DefaultAzureCredential: managed identity, workload identity, environment variables or az login), which needs a data role such as Storage Blob Data Contributor on the account or container. Returned URLs are https://<account>.blob.core.windows.net/<container>/<path>, readable only if the container allows anonymous blob access, which new storage accounts turn off; set public_url for Azure Front Door or a CDN. get_url with expires returns a SAS URL, signed with the account key, or under an Azure identity with a user delegation key (the identity needs the Storage Blob Delegator role; expires is then at most 7 days). A connection string with a SAS token instead of an AccountKey cannot sign URLs. Uploads set the blob’s content type, and deleting a missing blob succeeds. The client’s connections are closed on app shutdown.

r2, spaces, cubbit, ovh and b2 have no default credential chain: both KIRAK_STORAGE_<INSTANCE>_ACCESS_KEY and KIRAK_STORAGE_<INSTANCE>_SECRET_KEY are required.

Public URLs of S3-type instances. Upload results, get_url without expires and list return <public_url>/<path> when public_url is set (a CDN or custom domain in front of the bucket), otherwise the bucket’s own URL. Kirak does not make a file readable: the URL works only if the bucket allows public reads, or if the upload is sent with "acl": "public-read" on a bucket that accepts ACLs (new AWS buckets do not: leave acl unset there). For private buckets use get_url with expires.

Default: a call without provider uses default_provider.
Per call: pass "provider": "<instance name>" in the params dict, or as a form field / query parameter on the HTTP routes. The value is the instance name (archive), never the type (aws).
Allow-list: only instances listed in kirak.json can be used. Anything else fails with PROVIDER_NOT_CONFIGURED (400).
Secrets are never read from kirak.json. Each S3-type instance reads its own credentials from KIRAK_STORAGE_<INSTANCE>_ACCESS_KEY and KIRAK_STORAGE_<INSTANCE>_SECRET_KEY, e.g. KIRAK_STORAGE_MAIN_ACCESS_KEY. For aws and wasabi, if they are unset, boto3 falls back to its default credential chain. Set both or neither: one without the other is a ConfigurationError. gcs and azure read their own secrets, described in their sections above.
Module-wide settings (image_*, file_*, default_thumbnails, thumbnail_mode) stay at the top level of "storage" and apply to every instance.

Remembering where a file lives. A stored path does not say which provider holds it. Upload responses include "provider" with the instance name, so store it next to the path. delete, get_url and list need that same provider for any file that is not on the default instance.

result = await kirak.storage.upload_file({"file": data, "path": "reports/q1.pdf", "provider": "archive"})
# result["data"]["provider"] == "archive"  -> save it with the path

await kirak.storage.get_url({"path": "reports/q1.pdf", "provider": "archive", "expires": 3600})

`upload_image`

Upload and process an image. Pipeline: validate -> compress -> resize -> upload original -> generate thumbnails.

result = await kirak.storage.upload_image({
    "file": file_bytes,                   # raw bytes from form upload
    "path": "images/avatars/user_42.jpg", # logical storage path
    "filename": "photo.jpg",              # used for format/extension detection
    "thumbnails": [
        {"name": "sm", "width": 100, "height": 100},
        {"name": "md", "width": 300, "height": 300},
        {"name": "lg", "width": 600, "height": 600},
    ],
    "background_tasks": background_tasks,  # FastAPI BackgroundTasks (for async thumbnails)
})

Parameters:

Parameter	Default	Description
`file`	–	Required. Raw image bytes.
`path`	–	Required. Storage path, e.g. `"images/articles/abc.jpg"`.
`filename`	`"image.jpg"`	Original filename – used for format detection.
`thumbnails`	`None`	`None` (no thumbnails), `True` (use `default_thumbnails` from kirak.json), or a list of size dicts.
`background_tasks`	`None`	FastAPI `BackgroundTasks` instance. Required for `thumbnail_mode=background`.

Response:

{
  "statusCode": 200,
  "status": "success",
  "message": "Image uploaded successfully",
  "data": {
    "original": "https://cdn.example.com/42/images/avatars/user_42.jpg",
    "thumbnails": {
      "sm": "https://cdn.example.com/42/images/avatars/user_42_sm.jpg",
      "md": "https://cdn.example.com/42/images/avatars/user_42_md.jpg",
      "lg": "https://cdn.example.com/42/images/avatars/user_42_lg.jpg"
    },
    "provider": "main"
  }
}

Failures: a missing file or path, a type not in image_allowed_types, content that does not match the extension, or a file over image_max_size raises ValidationError (400). An unknown provider raises PROVIDER_NOT_CONFIGURED (400). A processing or provider failure raises INTERNAL_ERROR (500).

Storage refuses a guest with AuthenticationError (401), on all five operations. Every /storage/* route runs as the signed-in caller. A call from your own code must set an identity first (see Who a direct call runs as): the request’s caller in your own route (set_user_context(caller)), or the system identity {"role": "system", "user_id": None, "token": None} in a job, a script or startup code. Without one the call runs as guest and is refused. This also keeps a guest MCP server or an agent run for a guest away from stored files.

For a caller with any role except admin, system and superadmin, Kirak prefixes the path (or list prefix) with {user_id}/ unless it already starts with it. path="images/avatars/user_42.jpg" from user 42 is stored at 42/images/avatars/user_42.jpg, which is what the returned URL reflects, and delete, get_url and list from user 42 reach only keys under 42/. admin, system and superadmin callers use the path as given. A caller of another role with no user id is refused with PermissionDenied (403) instead of being left unscoped. A path containing a .. segment is refused with ValidationError (400) for every caller.

Image Processing

Before upload, Kirak:

Validates the file extension against image_allowed_types (default "jpg,jpeg,png,webp,gif")
Checks that the content matches the extension (with python-magic when installed, otherwise by the file’s header bytes)
Validates file size against image_max_size (default "5mb")
Compresses JPEG/WebP images to image_compress_quality (default 85)
Scales the image down to fit image_max_width x image_max_height (default 2000 x 2000), preserving aspect ratio; smaller images are never enlarged

These are kirak.json manifest settings under the storage section, not environment variables.

Animated GIF and WebP images skip steps 4 and 5 and are stored as uploaded, so they keep every frame; their thumbnails show the first frame. SVG cannot be processed: adding svg to image_allowed_types does not make it an image type (it is refused with a 400). Store vector files with upload_file instead, and be aware that an SVG served from your own domain (the local provider) can run script in your users’ browsers.

Thumbnail Modes

Mode	Behaviour
`sync` (default)	Thumbnails generated before the response is returned. Simpler, slightly slower responses.
`background`	Response returned immediately; thumbnails generated in a FastAPI `BackgroundTask`. Pass `background_tasks=...` from the route; without it, thumbnails are generated synchronously.

In background mode the response has "processing": true and the message "Image uploaded; thumbnails processing". The thumbnails URLs are the ones the thumbnails will have: they return 404 until the background task has stored them.

{ "storage": { "thumbnail_mode": "background" } }

Default Thumbnails

Set global thumbnail sizes that apply to every upload_image({"thumbnails": True}) call, in kirak.json:

{ "storage": { "default_thumbnails": "sm:100x100,md:300x300,lg:600x600" } }

In a FastAPI Route

from fastapi import UploadFile, File, BackgroundTasks, Request
from kirak.core.context import set_user_context, reset_user_context

@app.post("/upload/avatar")
async def upload_avatar(
    request: Request,
    background_tasks: BackgroundTasks,
    file: UploadFile = File(...),
):
    kirak = request.app.state.kirak
    caller = (await kirak.auth.get_current_user({"request": request})).get("data") or {}
    file_bytes = await file.read()
    token = set_user_context(caller)
    try:
        result = await kirak.storage.upload_image({
            "file": file_bytes,
            "path": "images/avatar.jpg",
            "filename": file.filename,
            "thumbnails": True,
            "background_tasks": background_tasks,
        })
        if result["status"] == "success":
            await kirak.update("users", {
                "where": {"id": caller["user_id"]},
                "data": {"avatar": result["data"]["original"]},
            })
    finally:
        reset_user_context(token)
    return result

The route sets the caller, so the image is stored under the caller’s prefix (42/images/avatar.jpg for user 42) and the users update runs under that user’s access rules. Without set_user_context(caller) the call runs as guest and storage refuses it with a 401.

`upload_file`

Upload a document or binary file without image processing.

result = await kirak.storage.upload_file({
    "file": file_bytes,
    "path": "files/documents/report_2025.pdf",
    "filename": "report_2025.pdf",
})

Response:

{
  "statusCode": 200,
  "status": "success",
  "message": "File uploaded successfully",
  "data": {
    "url": "https://cdn.example.com/files/documents/report_2025.pdf",
    "filename": "report_2025.pdf",
    "size": 204800,
    "mime_type": "application/pdf",
    "provider": "main"
  }
}

Validates extension against file_allowed_types (default "pdf,doc,docx,xls,xlsx,csv,txt,zip") and size against file_max_size (default "50mb") – both kirak.json manifest settings, not environment variables. The returned url is also subject to the same per-user path scoping described above. Failures are the same as for upload_image.

`delete`

Delete a stored file. Optionally delete thumbnail variants.

result = await kirak.storage.delete({
    "path": "images/avatars/user_42.jpg",
    "with_thumbnails": True,  # also deletes user_42_sm.jpg, user_42_md.jpg, etc.
})
# -> {"status": "success", "data": {"path": "images/avatars/user_42.jpg"}, ...}

data.path is the key that was deleted, after per-user scoping (42/images/avatars/user_42.jpg for user 42).

A failure raises KirakException (STORAGE_ERROR, HTTP 500) instead of returning an error result.

When with_thumbnails=True, Kirak reads default_thumbnails (kirak.json) to determine which thumbnail paths to delete. Thumbnail deletions are best-effort – a missing thumbnail does not fail the operation.

`get_url`

Get the public URL for a stored file.

result = await kirak.storage.get_url({"path": "images/avatars/user_42.jpg"})
url = result["data"]["url"]

# Presigned URL (cloud instances only -- expires in 3600 seconds)
result = await kirak.storage.get_url({
    "path": "files/documents/private_report.pdf",
    "expires": 3600,
})

For a local instance, returns a URL under /media/<instance name>/. For an S3-type instance without expires, returns the permanent public URL (under public_url when set). With expires, returns a presigned URL that grants temporary access to a non-public object; it always points at the bucket endpoint, never at public_url.

`list`

List files under a path prefix.

result = await kirak.storage.list({"prefix": "images/articles/"})
files = result["data"]["files"]
# -> [
#      { "path": "images/articles/abc.jpg", "size": 48000, "url": "https://..." },
#      ...
#    ]

Returns every file under the prefix in one list; on S3-type instances Kirak follows all result pages.

Local Static File Serving

Every instance of type local mounts its upload_dir as a FastAPI StaticFiles endpoint at /media/<instance name>, whether or not it is the default. For an instance named local with upload_dir assets/media, the file assets/media/images/foo.jpg is served at http://localhost:8000/media/local/images/foo.jpg. Two local instances must use different upload_dir values. Instances of every other type are not created at mount time, only on first use.

Hooks

@kirak.storage.hook("after_upload_image")
async def after_image_upload(result):
    if result.get("status") == "success":
        # Run content moderation, update index, etc.
        image_url = result["data"]["original"]
    return result

Every operation fires before_<operation> and after_<operation>: upload_image, upload_file, delete, get_url and list (for example after_delete). A before_ hook receives the params as passed, before per-user scoping; the after_ hook receives the result envelope.

HTTP Endpoints

Every endpoint requires an Authorization: Bearer <access token> header. A missing token fails with MISSING_TOKEN (401), and a revoked token is rejected when the auth module is on.

Method	Path	Auth	Description
POST	`/storage/upload/image`	bearer	Upload and process an image
POST	`/storage/upload/file`	bearer	Upload a document or binary file
DELETE	`/storage/delete`	bearer	Delete a file
GET	`/storage/url`	bearer	Get public URL for a file
GET	`/storage/list`	bearer	List files under a prefix

Adding a Custom Storage Provider

The shared contract, the credential check (check()), entry points and testing are covered in Adding a Provider.

Subclass StorageProvider (kirak/storage/providers/base.py), register the type, and list an instance in kirak.json.

from kirak.storage.providers.base import StorageProvider


class AcmeProvider(StorageProvider):
    TYPE_NAME = "acme"                      # the "type" in kirak.json
    SECRET_FIELDS = ("api_key",)            # read from KIRAK_STORAGE_<INSTANCE>_API_KEY

    def __init__(self, config, storage=None):
        super().__init__(config, storage)
        self._require("bucket")             # fail early on missing config
        ...

    async def upload(self, file_bytes, path, content_type="application/octet-stream") -> str: ...
    async def delete(self, path) -> bool: ...
    async def get_url(self, path, expires=None) -> str: ...
    async def list_files(self, prefix="") -> list: ...
    async def close(self) -> None: ...      # optional: called on app shutdown, e.g. to close a session

async def on_kirak_ready(kirak):
    kirak.storage.register_provider("acme", AcmeProvider)

"storage": {
  "default_provider": "acme",
  "providers": { "acme": { "type": "acme", "bucket": "my-bucket" } }
}

register_provider also works as a decorator, raises ValueError if the type name is taken (including by a built-in), and TypeError if the class does not subclass StorageProvider. A provider can instead be published as a pip package with an entry point in the kirak.storage_providers group, so users need only pip install plus a type in kirak.json:

[project.entry-points."kirak.storage_providers"]
acme = "your_package.acme:AcmeProvider"

To contribute a provider to Kirak core, add the class under kirak/storage/services/, declare it with a ProviderSpec in kirak/catalog/specs/storage.py (the built-in table is built from it; see Adding a Provider), and add tests.