S3 Event store implementation

Very simplistic system to store events on S3

S3EventSource - Individual objects

This stores events one-per-object

const eventSource = new S3EventSource({
 s3: s3client, // optional
 bucket: "s3-bucket-name",
 prefix: "taxonomy/" // optional
})

S3PackedEventSource - Single ZIP file

This reads (only) events cached in a ZIP file

const eventSource = new S3PackedEventSource({
 s3: s3client, // optional
 bucket: "s3-bucket-name",
 prefix: "taxonomy/events.zip" // optional
})

This event source should be closed to clean up temporary files after use.

S3PackSequenceEventSource - Sequence of ZIP files

This reads events cached in a sequence of ZIP files, and permits writing through a batch interface.

const eventSource = new S3PackSequenceEventSource({
 s3: s3client, // optional
 bucket: "s3-bucket-name",
 prefix: "taxonomy/" // optional
})
await useBatchWriter(eventSource.batchWriter, async writer => {
  await writer.writeEvent({category: "example", id: "12345"}, [{data: true}])
  await writer.flush()
})

Tests

Running the tests requires that the default AWS credentials be able to write to a bucket called "s3-eventstore-testing". This is set up for DevTeam-InfraLogic in inframationdev.