# @push.rocks/smartstorage

create an S3-compatible endpoint that map to a local directory.

# readme.md for @push.rocks/smartstorage

A high-performance, S3-compatible storage server powered by a **Rust core** with a clean TypeScript API. Runs standalone for dev/test — or scales out as a **distributed, erasure-coded cluster** with QUIC-based inter-node communication. No cloud, no Docker. Just `npm install` and go. 🚀

## Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit [community.foss.global/](https://community.foss.global/). This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a [code.foss.global/](https://code.foss.global/) account to submit Pull Requests directly.

## Why smartstorage?

| Feature | smartstorage | MinIO | s3rver |
|---------|-------------|-------|--------|
| Install | `pnpm add` | Docker / binary | `npm install` |
| Startup time | ~20ms | seconds | ~200ms |
| Large file uploads | Streaming, zero-copy | Yes | OOM risk |
| Range requests | Seek-based | Yes | Full read |
| Language | Rust + TypeScript | Go | JavaScript |
| Multipart uploads | ✅ Full support | Yes | No |
| Auth | AWS SigV4 (full verification) | Full IAM | Basic |
| Bucket policies | IAM-style evaluation | Yes | No |
| Clustering | ✅ Erasure-coded, QUIC | Yes | No |
| Multi-drive awareness | ✅ Per-drive health | Yes | No |

### Core Features

- 🦀 **Rust-powered HTTP server** — hyper 1.x with streaming I/O, zero-copy, backpressure
- 📦 **Full S3-compatible API** — works with AWS SDK v3, SmartBucket, any S3 client
- 💾 **Filesystem-backed storage** — buckets map to directories, objects to files
- 📤 **Streaming multipart uploads** — large files without memory pressure
- 📐 **Byte-range requests** — `seek()` directly to the requested byte offset
- 🔐 **AWS SigV4 authentication** — full signature verification with constant-time comparison
- 📋 **Bucket policies** — IAM-style JSON policies with Allow/Deny evaluation and wildcard matching
- 🌐 **CORS middleware** — configurable cross-origin support
- 🧹 **Clean slate mode** — wipe storage on startup for test isolation
- ⚡ **Test-first design** — start/stop in milliseconds, no port conflicts

### Clustering Features

- 🔗 **Erasure coding** — Reed-Solomon (configurable k data + m parity shards) for storage efficiency and fault tolerance
- 🚄 **QUIC transport** — multiplexed, encrypted inter-node communication via `quinn` with zero head-of-line blocking
- 💽 **Multi-drive awareness** — each node manages multiple independent storage paths with health monitoring
- 🤝 **Cluster membership** — static seed config + runtime join, heartbeat-based failure detection
- ✍️ **Quorum writes** — data is only acknowledged after k+1 shards are persisted
- 📖 **Quorum reads** — reconstruct from any k available shards, local-first fast path
- 🩹 **Self-healing** — background scanner detects and reconstructs missing/corrupt shards

## Installation

```bash
pnpm add @push.rocks/smartstorage -D
```

> **Note:** The package ships with precompiled Rust binaries for `linux_amd64` and `linux_arm64`. No Rust toolchain needed on your machine.

## Quick Start

### Standalone Mode (Dev & Test)

```typescript
import { SmartStorage } from '@push.rocks/smartstorage';

// Start a local S3-compatible storage server
const storage = await SmartStorage.createAndStart({
  server: { port: 3000 },
  storage: { cleanSlate: true },
});

// Create a bucket
await storage.createBucket('my-bucket');

// Get connection details for any S3 client
const descriptor = await storage.getStorageDescriptor();
// → { endpoint: 'localhost', port: 3000, accessKey: 'STORAGE', accessSecret: 'STORAGE', useSsl: false }

// When done
await storage.stop();
```

### Cluster Mode (Distributed)

```typescript
import { SmartStorage } from '@push.rocks/smartstorage';

const storage = await SmartStorage.createAndStart({
  server: { port: 3000 },
  cluster: {
    enabled: true,
    nodeId: 'node-1',
    quicPort: 4000,
    seedNodes: ['192.168.1.11:4000', '192.168.1.12:4000'],
    erasure: {
      dataShards: 4,      // k: minimum shards to reconstruct data
      parityShards: 2,    // m: fault tolerance (can lose up to m shards)
    },
    drives: {
      paths: ['/mnt/disk1', '/mnt/disk2', '/mnt/disk3'],
    },
  },
});
```

Objects are automatically split into chunks (default 4 MB), erasure-coded into 6 shards (4 data + 2 parity), and distributed across drives/nodes. Any 4 of 6 shards can reconstruct the original data.

## Configuration

All config fields are optional — sensible defaults are applied automatically.

```typescript
import { SmartStorage, ISmartStorageConfig } from '@push.rocks/smartstorage';

const config: ISmartStorageConfig = {
  server: {
    port: 3000,              // Default: 3000
    address: '0.0.0.0',      // Default: '0.0.0.0'
    silent: false,           // Default: false
    region: 'us-east-1',     // Default: 'us-east-1' — used for SigV4 signing
  },
  storage: {
    directory: './my-data',  // Default: .nogit/bucketsDir
    cleanSlate: false,       // Default: false — set true to wipe on start
  },
  auth: {
    enabled: false,          // Default: false
    credentials: [{
      accessKeyId: 'MY_KEY',
      secretAccessKey: 'MY_SECRET',
    }],
  },
  cors: {
    enabled: false,          // Default: false
    allowedOrigins: ['*'],
    allowedMethods: ['GET', 'POST', 'PUT', 'DELETE', 'HEAD', 'OPTIONS'],
    allowedHeaders: ['*'],
    exposedHeaders: ['ETag', 'x-amz-request-id', 'x-amz-version-id'],
    maxAge: 86400,
    allowCredentials: false,
  },
  logging: {
    level: 'info',           // 'error' | 'warn' | 'info' | 'debug'
    format: 'text',          // 'text' | 'json'
    enabled: true,
  },
  limits: {
    maxObjectSize: 5 * 1024 * 1024 * 1024, // 5 GB
    maxMetadataSize: 2048,
    requestTimeout: 300000,  // 5 minutes
  },
  multipart: {
    expirationDays: 7,
    cleanupIntervalMinutes: 60,
  },
  cluster: {                 // Optional — omit for standalone mode
    enabled: true,
    nodeId: 'node-1',        // Auto-generated UUID if omitted
    quicPort: 4000,          // Default: 4000
    seedNodes: [],           // Addresses of existing cluster members
    erasure: {
      dataShards: 4,         // Default: 4
      parityShards: 2,       // Default: 2
      chunkSizeBytes: 4194304, // Default: 4 MB
    },
    drives: {
      paths: ['/mnt/disk1', '/mnt/disk2'],
    },
    heartbeatIntervalMs: 5000,  // Default: 5000
    heartbeatTimeoutMs: 30000,  // Default: 30000
  },
};

const storage = await SmartStorage.createAndStart(config);
```

### Common Configurations

**CI/CD testing** — silent, clean, fast:
```typescript
const storage = await SmartStorage.createAndStart({
  server: { port: 9999, silent: true },
  storage: { cleanSlate: true },
});
```

**Auth enabled:**
```typescript
const storage = await SmartStorage.createAndStart({
  auth: {
    enabled: true,
    credentials: [{ accessKeyId: 'test', secretAccessKey: 'test123' }],
  },
});
```

**CORS for local web dev:**
```typescript
const storage = await SmartStorage.createAndStart({
  cors: {
    enabled: true,
    allowedOrigins: ['http://localhost:5173'],
    allowCredentials: true,
  },
});
```

## Usage with AWS SDK v3

```typescript
import { S3Client, PutObjectCommand, GetObjectCommand, DeleteObjectCommand } from '@aws-sdk/client-s3';

const descriptor = await storage.getStorageDescriptor();

const client = new S3Client({
  endpoint: `http://${descriptor.endpoint}:${descriptor.port}`,
  region: 'us-east-1',
  credentials: {
    accessKeyId: descriptor.accessKey,
    secretAccessKey: descriptor.accessSecret,
  },
  forcePathStyle: true,  // Required for path-style access
});

// Upload
await client.send(new PutObjectCommand({
  Bucket: 'my-bucket',
  Key: 'hello.txt',
  Body: 'Hello, Storage!',
  ContentType: 'text/plain',
}));

// Download
const { Body } = await client.send(new GetObjectCommand({
  Bucket: 'my-bucket',
  Key: 'hello.txt',
}));
const content = await Body.transformToString(); // "Hello, Storage!"

// Delete
await client.send(new DeleteObjectCommand({
  Bucket: 'my-bucket',
  Key: 'hello.txt',
}));
```

## Usage with SmartBucket

```typescript
import { SmartBucket } from '@push.rocks/smartbucket';

const smartbucket = new SmartBucket(await storage.getStorageDescriptor());
const bucket = await smartbucket.createBucket('my-bucket');
const dir = await bucket.getBaseDirectory();

// Upload
await dir.fastPut({ path: 'docs/readme.txt', contents: 'Hello!' });

// Download
const content = await dir.fastGet('docs/readme.txt');

// List
const files = await dir.listFiles();
```

## Multipart Uploads

For files larger than 5 MB, use multipart uploads. smartstorage handles them with **streaming I/O** — parts are written directly to disk, never buffered in memory. In cluster mode, each part is independently erasure-coded and distributed.

```typescript
import {
  CreateMultipartUploadCommand,
  UploadPartCommand,
  CompleteMultipartUploadCommand,
} from '@aws-sdk/client-s3';

// 1. Initiate
const { UploadId } = await client.send(new CreateMultipartUploadCommand({
  Bucket: 'my-bucket',
  Key: 'large-file.bin',
}));

// 2. Upload parts
const parts = [];
for (let i = 0; i < chunks.length; i++) {
  const { ETag } = await client.send(new UploadPartCommand({
    Bucket: 'my-bucket',
    Key: 'large-file.bin',
    UploadId,
    PartNumber: i + 1,
    Body: chunks[i],
  }));
  parts.push({ PartNumber: i + 1, ETag });
}

// 3. Complete
await client.send(new CompleteMultipartUploadCommand({
  Bucket: 'my-bucket',
  Key: 'large-file.bin',
  UploadId,
  MultipartUpload: { Parts: parts },
}));
```

## Bucket Policies

smartstorage supports AWS-style bucket policies for fine-grained access control. Policies use the same IAM JSON format as real S3 — so you can develop and test your policy logic locally before deploying.

When `auth.enabled` is `true`, the auth pipeline works as follows:
1. **Authenticate** — verify the AWS SigV4 signature (anonymous requests skip this step)
2. **Authorize** — evaluate bucket policies against the request action, resource, and caller identity
3. **Default** — authenticated users get full access; anonymous requests are denied unless a policy explicitly allows them

### Setting a Bucket Policy

```typescript
import { PutBucketPolicyCommand } from '@aws-sdk/client-s3';

// Allow anonymous read access to all objects in a bucket
await client.send(new PutBucketPolicyCommand({
  Bucket: 'public-assets',
  Policy: JSON.stringify({
    Version: '2012-10-17',
    Statement: [{
      Sid: 'PublicRead',
      Effect: 'Allow',
      Principal: '*',
      Action: ['s3:GetObject'],
      Resource: ['arn:aws:s3:::public-assets/*'],
    }],
  }),
}));
```

### Policy Features

- **Effect**: `Allow` and `Deny` (explicit Deny always wins)
- **Principal**: `"*"` (everyone) or `{ "AWS": ["arn:..."] }` for specific identities
- **Action**: IAM-style actions like `s3:GetObject`, `s3:PutObject`, `s3:*`, or prefix wildcards like `s3:Get*`
- **Resource**: ARN patterns with `*` and `?` wildcards (e.g. `arn:aws:s3:::my-bucket/*`)
- **Persistence**: Policies survive server restarts — stored as JSON on disk alongside your data

### Policy CRUD Operations

| Operation | AWS SDK Command | HTTP |
|-----------|----------------|------|
| Get policy | `GetBucketPolicyCommand` | `GET /{bucket}?policy` |
| Set policy | `PutBucketPolicyCommand` | `PUT /{bucket}?policy` |
| Delete policy | `DeleteBucketPolicyCommand` | `DELETE /{bucket}?policy` |

Deleting a bucket automatically removes its associated policy.

## Clustering Deep Dive 🔗

smartstorage can run as a distributed storage cluster where multiple nodes cooperate to store and retrieve data with built-in redundancy.

### How It Works

```
Client ──HTTP PUT──▶ Node A (coordinator)
                       │
                       ├─ Split object into 4 MB chunks
                       ├─ Erasure-code each chunk (4 data + 2 parity = 6 shards)
                       │
                       ├──QUIC──▶ Node B (shard writes)
                       ├──QUIC──▶ Node C (shard writes)
                       └─ Local disk (shard writes)
```

1. **Any node can coordinate** — the client connects to any cluster member
2. **Objects are chunked** — large objects split into fixed-size pieces (default 4 MB)
3. **Each chunk is erasure-coded** — Reed-Solomon produces k data + m parity shards
4. **Shards are distributed** — placed across different nodes and drives for fault isolation
5. **Quorum guarantees consistency** — writes need k+1 acks, reads need k shards

### Erasure Coding

With the default `4+2` configuration:
- Storage overhead: **33%** (vs. 200% for 3x replication)
- Fault tolerance: **any 2 drives/nodes can fail** simultaneously
- Read efficiency: only **4 of 6 shards** needed to reconstruct data

| Config | Total Shards | Overhead | Tolerance | Min Nodes |
|--------|-------------|----------|-----------|-----------|
| 4+2 | 6 | 33% | 2 failures | 3 |
| 6+3 | 9 | 50% | 3 failures | 5 |
| 2+1 | 3 | 50% | 1 failure | 2 |

### QUIC Transport

Inter-node communication uses [QUIC](https://en.wikipedia.org/wiki/QUIC) via the `quinn` library:
- 🔒 **Built-in TLS** — self-signed certs auto-generated at cluster init
- 🔀 **Multiplexed streams** — concurrent shard transfers without head-of-line blocking
- ⚡ **Connection pooling** — persistent connections to peer nodes
- 🌊 **Natural backpressure** — QUIC flow control prevents overloading slow peers

### Cluster Membership

- **Static seed nodes** — initial cluster defined in config
- **Runtime join** — new nodes can join a running cluster
- **Heartbeat monitoring** — every 5s (configurable), with suspect/offline detection
- **Split-brain prevention** — nodes only mark peers offline when they have majority

### Self-Healing

A background scanner periodically (default: every 24h):
1. Checks shard checksums (CRC32C) for bit-rot detection
2. Identifies shards on offline nodes
3. Reconstructs missing shards from remaining data using Reed-Solomon
4. Places healed shards on healthy drives

Healing runs at low priority to avoid impacting foreground I/O.

### Erasure Set Formation

Drives are organized into fixed **erasure sets** at cluster initialization:

```
3 nodes × 4 drives each = 12 drives total
With 6-shard erasure sets → 2 erasure sets

Set 0: Node1-Disk0, Node2-Disk0, Node3-Disk0, Node1-Disk1, Node2-Disk1, Node3-Disk1
Set 1: Node1-Disk2, Node2-Disk2, Node3-Disk2, Node1-Disk3, Node2-Disk3, Node3-Disk3
```

Drives are interleaved across nodes for maximum fault isolation. New nodes form new erasure sets — existing data is never rebalanced.

## Testing Integration

```typescript
import { SmartStorage } from '@push.rocks/smartstorage';
import { tap, expect } from '@git.zone/tstest/tapbundle';

let storage: SmartStorage;

tap.test('setup', async () => {
  storage = await SmartStorage.createAndStart({
    server: { port: 4567, silent: true },
    storage: { cleanSlate: true },
  });
});

tap.test('should store and retrieve objects', async () => {
  await storage.createBucket('test');
  // ... your test logic using AWS SDK or SmartBucket
});

tap.test('teardown', async () => {
  await storage.stop();
});

export default tap.start();
```

## API Reference

### `SmartStorage` Class

#### `static createAndStart(config?: ISmartStorageConfig): Promise<SmartStorage>`

Create and start a server in one call.

#### `start(): Promise<void>`

Spawn the Rust binary and start the HTTP server.

#### `stop(): Promise<void>`

Gracefully stop the server and kill the Rust process.

#### `createBucket(name: string): Promise<{ name: string }>`

Create a storage bucket.

#### `getStorageDescriptor(options?): Promise<IS3Descriptor>`

Get connection details for S3-compatible clients. Returns:

| Field | Type | Description |
|-------|------|-------------|
| `endpoint` | `string` | Server hostname (`localhost` by default) |
| `port` | `number` | Server port |
| `accessKey` | `string` | Access key from first configured credential |
| `accessSecret` | `string` | Secret key from first configured credential |
| `useSsl` | `boolean` | Always `false` (plain HTTP) |

## Architecture

smartstorage uses a **hybrid Rust + TypeScript** architecture:

```
┌──────────────────────────────────────────────┐
│  Your Code (AWS SDK, SmartBucket, etc.)       │
│  ↕ HTTP (localhost:3000)                     │
├──────────────────────────────────────────────┤
│  ruststorage binary (Rust)                    │
│  ├─ hyper 1.x HTTP server                   │
│  ├─ S3 path-style routing                   │
│  ├─ StorageBackend (Standalone or Clustered) │
│  │   ├─ FileStore (single-node mode)        │
│  │   └─ DistributedStore (cluster mode)     │
│  │       ├─ ErasureCoder (Reed-Solomon)     │
│  │       ├─ ShardStore (per-drive storage)  │
│  │       ├─ QuicTransport (quinn)           │
│  │       ├─ ClusterState & Membership       │
│  │       └─ HealingService                  │
│  ├─ SigV4 auth + policy engine              │
│  ├─ CORS middleware                          │
│  └─ S3 XML response builder                 │
├──────────────────────────────────────────────┤
│  TypeScript (thin IPC wrapper)               │
│  ├─ SmartStorage class                       │
│  ├─ RustBridge (stdin/stdout JSON IPC)       │
│  └─ Config & S3 descriptor                  │
└──────────────────────────────────────────────┘
```

**Why Rust?** The original TypeScript implementation had critical perf issues: OOM on multipart uploads (parts buffered in memory), double stream copying, file descriptor leaks on HEAD requests, full-file reads for range requests, and no backpressure. The Rust binary solves all of these with streaming I/O, zero-copy, and direct `seek()` for range requests.

**IPC Protocol:** TypeScript spawns the `ruststorage` binary with `--management` and communicates via newline-delimited JSON over stdin/stdout. Commands: `start`, `stop`, `createBucket`, `clusterStatus`.

### S3-Compatible Operations

| Operation | Method | Path |
|-----------|--------|------|
| ListBuckets | `GET /` | |
| CreateBucket | `PUT /{bucket}` | |
| DeleteBucket | `DELETE /{bucket}` | |
| HeadBucket | `HEAD /{bucket}` | |
| ListObjects (v1/v2) | `GET /{bucket}` | `?list-type=2` for v2 |
| PutObject | `PUT /{bucket}/{key}` | |
| GetObject | `GET /{bucket}/{key}` | Supports `Range` header |
| HeadObject | `HEAD /{bucket}/{key}` | |
| DeleteObject | `DELETE /{bucket}/{key}` | |
| CopyObject | `PUT /{bucket}/{key}` | `x-amz-copy-source` header |
| InitiateMultipartUpload | `POST /{bucket}/{key}?uploads` | |
| UploadPart | `PUT /{bucket}/{key}?partNumber&uploadId` | |
| CompleteMultipartUpload | `POST /{bucket}/{key}?uploadId` | |
| AbortMultipartUpload | `DELETE /{bucket}/{key}?uploadId` | |
| ListMultipartUploads | `GET /{bucket}?uploads` | |
| GetBucketPolicy | `GET /{bucket}?policy` | |
| PutBucketPolicy | `PUT /{bucket}?policy` | |
| DeleteBucketPolicy | `DELETE /{bucket}?policy` | |

### On-Disk Format

**Standalone mode:**
```
{storage.directory}/
  {bucket}/
    {key}._storage_object                # Object data
    {key}._storage_object.metadata.json  # Metadata (content-type, x-amz-meta-*, etc.)
    {key}._storage_object.md5            # Cached MD5 hash
  .multipart/
    {upload-id}/
      metadata.json                      # Upload metadata
      part-1, part-2, ...               # Part data files
  .policies/
    {bucket}.policy.json                 # Bucket policy (IAM JSON format)
```

**Cluster mode:**
```
{drive_path}/.smartstorage/
  format.json                            # Drive metadata (cluster ID, erasure set)
  data/{bucket}/{key_hash}/{key}/
    chunk-{N}/shard-{M}.dat              # Erasure-coded shard data
    chunk-{N}/shard-{M}.meta             # Shard metadata (checksum, size)

{storage.directory}/
  .manifests/{bucket}/
    {key}.manifest.json                  # Object manifest (shard placements, checksums)
  .buckets/{bucket}/                     # Bucket metadata
  .policies/{bucket}.policy.json         # Bucket policies
```

## Related Packages

- [`@push.rocks/smartbucket`](https://code.foss.global/push.rocks/smartbucket) — High-level S3-compatible abstraction layer
- [`@push.rocks/smartrust`](https://code.foss.global/push.rocks/smartrust) — TypeScript ↔ Rust IPC bridge
- [`@git.zone/tsrust`](https://code.foss.global/git.zone/tsrust) — Rust cross-compilation for npm packages

## License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the [license](./license) file.

**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

### Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

### Company Information

Task Venture Capital GmbH
Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at hello@task.vc.

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.

# changelog.md for @push.rocks/smartstorage

## 2026-03-23 - 6.3.2 - fix(docs)
update license ownership and correct README license file reference

- Adjusts the copyright holder name in the license file
- Fixes the README link to match the lowercase license filename

## 2026-03-21 - 6.3.1 - fix(cluster)
improve shard reconstruction validation and start background healing service

- use the erasure read quorum when reconstructing chunks instead of assuming data shard count
- verify reconstructed shards before writing healed data back to disk
- start the healing service during server initialization with shared local shard stores
- simplify QUIC request handling by decoding the full request buffer including trailing shard data
- clean up unused variables and imports across cluster modules

## 2026-03-21 - 6.3.0 - feat(readme)
document distributed cluster mode, erasure coding, and QUIC-based architecture

- Expand README overview and feature matrix to highlight clustering, multi-drive awareness, and distributed storage capabilities
- Add standalone and cluster mode usage examples plus cluster configuration options
- Document clustering internals including erasure coding, quorum behavior, QUIC transport, self-healing, and on-disk layout

## 2026-03-21 - 6.2.0 - feat(cluster)
add shard healing, drive health heartbeats, and clustered policy directory support

- implements manifest-based healing that scans affected shards on offline nodes, reconstructs data with erasure coding, and rewrites recovered shards to local storage
- includes drive status reporting in membership heartbeats by wiring DriveManager health checks into cluster heartbeat messages
- adds clustered policies directory initialization and exposes policy storage paths from the distributed coordinator
- extends distributed coordinator support for remote shard read and delete operations plus multipart upload session metadata

## 2026-03-21 - 6.1.0 - feat(cluster)
add clustered storage backend with QUIC transport, erasure coding, and shard management

- introduces cluster configuration in Rust and TypeScript, including seed nodes, drive paths, heartbeat settings, and erasure coding options
- adds core cluster modules for membership, topology state, object manifests, placement, shard storage, drive management, healing scaffolding, and inter-node protocol handling
- adds QUIC-based transport for cluster communication and integrates a distributed storage backend alongside the existing standalone FileStore
- updates the server startup path to initialize standalone or clustered storage based on configuration and exposes a basic clusterStatus management endpoint
- refreshes build and dependency versions to support the new clustered storage implementation

## 2026-03-14 - 6.0.1 - fix(rust-bridge)
update smartrust and limit RustBridge binary lookup to dist_rust

- Bumps @push.rocks/smartrust from ^1.0.0 to ^1.3.2.
- Removes rust target debug and release fallback paths from RustBridge local binary resolution, relying on dist_rust/ruststorage.

## 2026-03-14 - 6.0.0 - BREAKING CHANGE(core)
Rebrand from smarts3 to smartstorage

- Package renamed from @push.rocks/smarts3 to @push.rocks/smartstorage
- Class renamed from Smarts3 to SmartStorage (no backward-compatible re-export)
- Interface renamed from ISmarts3Config to ISmartStorageConfig
- Method renamed from getS3Descriptor to getStorageDescriptor
- Rust binary renamed from rusts3 to ruststorage
- Rust types renamed: S3Error→StorageError, S3Action→StorageAction, S3Config→SmartStorageConfig, S3Server→StorageServer
- On-disk file extension changed from ._S3_object to ._storage_object (BREAKING for existing stored data)
- Default credentials changed from S3RVER to STORAGE
- All internal S3 branding removed; AWS S3 protocol compatibility (IAM actions, ARNs, SigV4) fully maintained

## 2026-02-17 - 5.3.0 - feat(auth)
add AWS SigV4 authentication and bucket policy support

- Implement AWS SigV4 full verification (constant-time comparison, 15-minute clock skew enforcement) and expose default signing region (server.region = 'us-east-1').
- Add IAM-style bucket policy engine with Put/Get/Delete policy APIs (GetBucketPolicy/PutBucketPolicy/DeleteBucketPolicy), wildcard action/resource matching, Allow/Deny evaluation, and on-disk persistence under .policies/{bucket}.policy.json.
- Documentation and README expanded with policy usage, examples, API table entries, and notes about policy CRUD and behavior for anonymous/authenticated requests.
- Rust code refactors: simplify storage/server result structs and multipart handling (removed several unused size/key/bucket fields), remove S3Error::to_response and error_xml helpers, and other internal cleanup to support new auth/policy features.

## 2026-02-17 - 5.2.0 - feat(auth,policy)
add AWS SigV4 authentication and S3 bucket policy support

- Implemented real AWS SigV4 verification (HMAC-SHA256), including x-amz-date handling, clock skew enforcement and constant-time signature comparison
- Added bucket policy model, validator and evaluation engine (Deny > Allow > NoOpinion) with a PolicyStore (RwLock cache + disk-backed .policies/*.policy.json)
- Integrated action resolution and auth+policy pipeline into the HTTP server: authorization checks run per-request, anonymous requests are denied by default, ListAllMyBuckets requires authentication
- Added bucket policy CRUD handlers via ?policy query parameter (GET/PUT/DELETE) and cleanup of policies on bucket deletion
- Storage and config updates: created .policies dir and policy path helpers; default region added to server config (TS + Rust)
- Added comprehensive tests for auth and policy behavior (policy CRUD, evaluation, per-action enforcement, auth integration)
- Updated Rust dependencies and Cargo.toml/Cargo.lock to include hmac, sha2, hex, subtle, cpufeatures

## 2026-02-13 - 5.1.1 - fix(smarts3)
replace TypeScript server with Rust-powered core and IPC bridge

- Major refactor: Node.js/TypeScript in-process server replaced by a Rust crate ('rusts3') with a TypeScript IPC wrapper (RustBridge).
- Removed many TypeScript server modules (smarts3-server, filesystem-store, multipart-manager, controllers, router, context, logger, xml utils, etc.); Smarts3Server export removed — public API now proxies to the Rust binary.
- Smarts3 now spawns and communicates with the rusts3 binary via RustBridge IPC (commands include start, stop, createBucket).
- Build & packaging changes: build script now runs `tsrust` before `tsbuild`; added `@git.zone/tsrust` devDependency; added `dist_rust` artifacts and new cross-compile targets in npmextra.json; .gitignore updated for rust/target.
- Dependency changes: added `@push.rocks/smartrust` (RustBridge) and simplified plugins surface; previous smartfs/smartxml usage removed from TS code and replaced by the Rust implementation + IPC.
- Added Rust project files (rust/Cargo.toml, rust/src/*) implementing server, IPC management loop, storage, XML responses, errors, and config.
- Documentation updated (README and hints) to describe the Rust core, supported prebuilt targets (linux_amd64, linux_arm64), IPC commands, and developer build notes.

## 2025-11-23 - 5.1.0 - feat(multipart)
Implement full multipart upload support with persistent manager, periodic cleanup, and API integration

- Add IMultipartConfig to server config with defaults (expirationDays: 7, cleanupIntervalMinutes: 60) and merge into existing config flow
- Introduce MultipartUploadManager: persistent upload metadata on disk, part upload/assembly, restore uploads on startup, listParts/listUploads, abort/cleanup functionality
- Start and stop multipart cleanup task from Smarts3Server lifecycle (startCleanupTask on start, stopCleanupTask on stop) with configurable interval and expiration
- ObjectController: support multipart endpoints (initiate, upload part, complete, abort) and move assembled final object into the object store on completion; set ETag headers and return proper XML responses
- BucketController: support listing in-progress multipart uploads via ?uploads query parameter and return S3-compatible XML
- Persist multipart state to disk and restore on initialization to survive restarts; perform automatic cleanup of expired uploads

## 2025-11-23 - 5.0.2 - fix(readme)
Clarify contribution agreement requirement in README

- Updated the Issue Reporting and Security section in readme.md to make it explicit that developers must sign and comply with the contribution agreement (and complete identification) before obtaining a code.foss.global account to submit pull requests.

## 2025-11-23 - 5.0.1 - fix(docs)
Clarify README wording about S3 compatibility and AWS SDK usage

- Update README wording to "Full S3 API compatibility" and clarify it works seamlessly with AWS SDK v3 and other S3 clients

## 2025-11-23 - 5.0.0 - BREAKING CHANGE(core)
Production-ready S3-compatible server: nested config, multipart uploads, CORS, structured logging, SmartFS migration and improved error handling

- Breaking change: configuration format migrated from flat to nested structure (server, storage, auth, cors, logging, limits). Update existing configs accordingly.
- Implemented full multipart upload support (initiate, upload part, complete, abort) with on-disk part management and final assembly.
- Added CORS middleware with configurable origins, methods, headers, exposed headers, maxAge and credentials support.
- Structured, configurable logging (levels: error|warn|info|debug; formats: text|json) and request/response logging middleware.
- Simple static credential authentication middleware (configurable list of credentials).
- Migrated filesystem operations to @push.rocks/smartfs (Web Streams interoperability) and removed smartbucket from production dependencies.
- Improved S3-compatible error handling and XML responses (S3Error class and XML utilities).
- Exposed Smarts3Server and made store/multipart managers accessible for tests and advanced usage; added helper methods like getS3Descriptor and createBucket.

## 2025-11-23 - 4.0.0 - BREAKING CHANGE(Smarts3)
Migrate Smarts3 configuration to nested server/storage objects and remove legacy flat config support

- Smarts3.createAndStart() and Smarts3 constructor now accept ISmarts3Config with nested `server` and `storage` objects.
- Removed support for the legacy flat config shape (top-level `port` and `cleanSlate`) / ILegacySmarts3Config.
- Updated tests to use new config shape (server:{ port, silent } and storage:{ cleanSlate }).
- mergeConfig and Smarts3Server now rely on the nested config shape; consumers must update their initialization code.

## 2025-11-23 - 3.2.0 - feat(multipart)
Add multipart upload support with MultipartUploadManager and controller integration

- Introduce MultipartUploadManager (ts/classes/multipart-manager.ts) to manage multipart upload lifecycle and store parts on disk
- Wire multipart manager into server and request context (S3Context, Smarts3Server) and initialize multipart storage on server start
- Add multipart-related routes and handlers in ObjectController: initiate (POST ?uploads), upload part (PUT ?partNumber&uploadId), complete (POST ?uploadId), and abort (DELETE ?uploadId)
- On complete, combine parts into final object and store via existing FilesystemStore workflow
- Expose multipart manager on Smarts3Server for controller access

## 2025-11-23 - 3.1.0 - feat(logging)
Add structured Logger and integrate into Smarts3Server; pass full config to server

- Introduce a new Logger class (ts/classes/logger.ts) providing leveled logging (error, warn, info, debug), text/json formats and an enable flag.
- Integrate Logger into Smarts3Server: use structured logging for server lifecycle events, HTTP request/response logging and S3 errors instead of direct console usage.
- Smarts3 now passes the full merged configuration into Smarts3Server (config.logging can control logging behavior).
- Server start/stop messages and internal request/error logs are emitted via the Logger and respect the configured logging level/format and silent option.

## 2025-11-23 - 3.0.4 - fix(smarts3)
Use filesystem store for bucket creation and remove smartbucket runtime dependency

- Switched createBucket to call the internal FilesystemStore.createBucket instead of using @push.rocks/smartbucket
- Made Smarts3Server.store public so Smarts3 can access the filesystem store directly
- Removed runtime import/export of @push.rocks/smartbucket from plugins and moved @push.rocks/smartbucket to devDependencies in package.json
- Updated createBucket to return a simple { name } object after creating the bucket via the filesystem store

## 2025-11-23 - 3.0.3 - fix(filesystem)
Migrate filesystem implementation to @push.rocks/smartfs and add Web Streams handling

- Replace dependency @push.rocks/smartfile with @push.rocks/smartfs and update README references
- plugins: instantiate SmartFs with SmartFsProviderNode and export smartfs (remove direct fs export)
- Refactor FilesystemStore to use smartfs directory/file APIs for initialize, reset, list, read, write, copy and delete
- Implement Web Stream ↔ Node.js stream conversion for uploads/downloads (Readable.fromWeb and writer.write with Uint8Array)
- Persist and read metadata (.metadata.json) and cached MD5 (.md5) via smartfs APIs
- Update readme.hints and documentation to note successful migration and next steps

## 2025-11-21 - 3.0.2 - fix(smarts3)
Prepare patch release 3.0.2 — no code changes detected

- No source changes in the diff
- Bump patch version from 3.0.1 to 3.0.2 for maintenance/release bookkeeping

## 2025-11-21 - 3.0.1 - fix(readme)
Add Issue Reporting and Security section to README

- Add guidance to report bugs, issues, and security vulnerabilities via community.foss.global
- Inform developers how to sign a contribution agreement and get a code.foss.global account to submit pull requests

## 2025-11-21 - 3.0.0 - BREAKING CHANGE(Smarts3)
Remove legacy s3rver backend, simplify Smarts3 server API, and bump dependencies

- Remove legacy s3rver backend: s3rver and its types were removed from dependencies and are no longer exported from plugins.
- Simplify Smarts3 API: removed useCustomServer option; Smarts3 now always uses the built-in Smarts3Server (s3Instance is Smarts3Server) and stop() always calls Smarts3Server.stop().
- Update README to remove legacy s3rver compatibility mention.
- Dependency updates: bumped @push.rocks/smartbucket to ^4.3.0 and @push.rocks/smartxml to ^2.0.0 (major upgrades), removed s3rver/@types/s3rver, bumped @aws-sdk/client-s3 to ^3.937.0 and @git.zone/tstest to ^3.1.0.

## 2025-11-21 - 2.3.0 - feat(smarts3-server)
Introduce native custom S3 server implementation (Smarts3Server) with routing, middleware, context, filesystem store, controllers and XML utilities; add SmartXml and AWS SDK test; keep optional legacy s3rver backend.

- Add Smarts3Server: native, Node.js http-based S3-compatible server (ts/classes/smarts3-server.ts)
- New routing and middleware system: S3Router and MiddlewareStack for pattern matching and middleware composition (ts/classes/router.ts, ts/classes/middleware-stack.ts)
- Introduce request context and helpers: S3Context for parsing requests, sending responses and XML (ts/classes/context.ts)
- Filesystem-backed storage: FilesystemStore with bucket/object operations, streaming uploads, MD5 handling and Windows-safe key encoding (ts/classes/filesystem-store.ts)
- S3 error handling: S3Error class that maps S3 error codes and produces XML error responses (ts/classes/s3-error.ts)
- Controllers for service, bucket and object operations with S3-compatible XML responses and copy/range support (ts/controllers/*.ts)
- XML utilities and SmartXml integration for consistent XML generation/parsing (ts/utils/xml.utils.ts, ts/plugins.ts)
- Expose native plugins (http, crypto, url, fs) and SmartXml via plugins.ts
- ts/index.ts: add useCustomServer option, default to custom server, export Smarts3Server and handle start/stop for both custom and legacy backends
- Add AWS SDK v3 integration test (test/test.aws-sdk.node.ts) to validate compatibility
- package.json: add @aws-sdk/client-s3 devDependency and @push.rocks/smartxml dependency
- Documentation: readme.md updated to describe native custom server and legacy s3rver compatibility

## 2025-11-20 - 2.2.7 - fix(core)
Update dependencies, code style and project config; add pnpm overrides and ignore AI folders

- Bump devDependencies and runtime dependencies (@git.zone/*, @push.rocks/*, @tsclass/tsclass, s3rver) to newer compatible versions
- Add pnpm.overrides entry to package.json and normalize repository URL format
- Code style and formatting fixes in TypeScript sources (ts/index.ts, ts/00_commitinfo_data.ts): whitespace, trailing commas, parameter formatting and minor API-return typing preserved
- tsconfig.json: simplify compiler options and compact exclude list
- Update .gitignore to add AI-related folders (.claude/, .serena/) to avoid accidental commits
- Documentation and changelog formatting tweaks (readme.md, changelog.md, npmextra.json) — whitespace/newline cleanups and expanded changelog entries

## 2025-08-16 - 2.2.6 - fix(Smarts3)

Allow overriding S3 descriptor; update dependencies, test config and documentation

- ts/index.ts: getS3Descriptor now accepts an optional Partial<IS3Descriptor> to override defaults (backwards compatible)
- package.json: updated devDependencies and runtime dependency versions (tstest, smartpath, tsclass, s3rver, etc.) and added packageManager field
- package.json: expanded test script to run tstest with --web --verbose --logfile --timeout 60
- test/test.ts: test instance port changed to 3333
- readme.md: major rewrite and expansion of usage examples, API reference and guides
- added project config files: .claude/settings.local.json and .serena/project.yml

## 2024-11-06 - 2.2.5 - fix(ci)

Corrected docker image URLs in Gitea workflows to match the correct domain format.

- Updated IMAGE environment variable in .gitea/workflows/default_nottags.yaml
- Updated IMAGE environment variable in .gitea/workflows/default_tags.yaml

## 2024-11-06 - 2.2.4 - fix(core)

Improve code style and update dependencies

- Updated @push.rocks/tapbundle to version ^5.4.3 in package.json.
- Fixed markdown formatting in readme.md.
- Improved code consistency in ts/00_commitinfo_data.ts, ts/plugins.ts, and test/test.ts.

## 2024-11-06 - 2.2.3 - fix(core)

Fix endpoint address from 'localhost' to '127.0.0.1' for better compatibility in Smarts3.getS3Descriptor

- Corrected the endpoint address in Smarts3.getS3Descriptor to ensure proper functioning across different environments.

## 2024-11-06 - 2.2.2 - fix(core)

Fixed function call for fastPut in the test suite to ensure proper file upload handling.

- Updated dependencies in package.json to newer versions.
- Corrected the function call in test suite for file upload.

## 2024-10-26 - 2.2.1 - fix(core)

Fix import and typings for improved compatibility

- Corrected the type signature for `getS3Descriptor` to return `IS3Descriptor`.
- Fixed import structure and updated dependencies for consistent namespace usage across plugins.

## 2024-10-26 - 2.2.0 - feat(ci)

Migrate CI/CD workflow from GitLab CI to Gitea CI

- Added new Gitea CI workflows for both non-tag and tag-based pushes
- Removed existing GitLab CI configuration

## 2024-05-29 - 2.1.1 - Updates and minor changes

Updates and changes based on minor configuration improvements and organizational shifts.

- Updated description file.
- Updated tsconfig settings.
- Updated npmextra.json with new githost configuration.
- Shifted to new organizational scheme.

## 2022-07-30 - 2.1.0 - Core improvements and fixes

Minor improvements and important core changes.

- Removed tslint from the core setup.

## 2022-07-30 - 2.0.2 - Bucket creation improvement

Enhanced file structure management.

- Improved bucket creation to store locally within the .nogit directory.

## 2022-04-14 - 2.0.0 to 2.0.1 - Structural updates and fixes

This release focused on core updates and structural changes.

- Reformatted the project structure.
- Core updates with minor fixes.

## 2021-12-20 - 1.0.10 - ESM Transition

Breaking changes and minor fixes, transitioning to ES Modules.

- BREAKING CHANGE: Transitioned core setup to ESM.