How it works
- The Miru Agent detects a new file to upload
- The Agent registers a new upload with the Miru control plane.
- Using credentials setup in the bucket connection, the control plane issues a short-lived token valid only for the exact storage object key the file will be uploaded to.
- The Agent uploads the file directly to your bucket.
File detection
The Agent uploads a file once it is confident the file has finished being written to. Completion is determined by quiescence: a file whose size and modification time have held steady for the rule’sstability_window_secs
is considered finished and eligible for upload.
While this works for any file type, it is a heuristic for detecting when a file is
finished being written to. For deterministic completion detection, write to a file location that does not match the rule’s glob and rename on completion. This ensures the Miru Agent only ever sees finished files — zero false positives.
Device data isolation
Devices’ uploads are strictly isolated from each other. Isolation rests on a single requirement on the destination path: it must include the{upload_id} variable somewhere in the template.
Since devices receive credentials scoped to an exact object key for each upload, and a given upload ID is authorized to its device, devices never receive credentials which grant any access to another device’s data.
Object metadata
Each uploaded object is stamped with the upload’s provenance as cloud object metadata (S3x-amz-meta-* / GCS x-goog-meta-*), so every file is self-describing in your
bucket and can be correlated back to exactly what produced it without parsing the
path.
The metadata is a generic map of string keys to string values. The keys currently
stamped are listed below, but the set may evolve. The map stays within the cloud
provider’s object-metadata size limits (roughly 2 KB on S3, 8 KB on GCS).
The ID of the device that uploaded the file.Example:
dvc_123The ID of the deployment that was running when the file was detected.Example:
dpl_123The ID of the release that was running when the file was detected.Example:
rls_1234567890The version field of the release that was running when the file was detected.Example:
v1.0.0The file’s
mtime timestamp as reported by the operating system.Example: 2021-01-01T00:00:00Z
