Erasing Data
Vendia Share allows both scalar and file data to be permanently and irrevocably erased while retaining the tamper-proof, verifiable, and immutable properties of the distributed ledger.
Erasure terms
Erasure
The process of erasing scalar and file data in Vendia Share.
Erasure metadata
Metadata stored on objects in world state and on entries in the ledger describing field erasure state as well as hash and salt metadata that can be used for verification.
Field
A field or property defined on an object in Vendia.
Global erasure
The process of erasing data across all nodes in a Uni.
Hash
The cryptographic hash of a field value in Vendia.
Local erasure
The process of erasing data on the local node only.
Salt
A randomly generated value used to hash a field in Vendia.
Token
The cryptographic hash used to replace plaintext fields in mutations in the ledger.
Tokenization
The process of replacing plaintext fields in a mutation with cryptographically hashed token values
Masking
The process of masking (redacting) previously erased fields so that they cannot be restored by subsequent mutations.
Vault
The datastore containing plaintext values for erasable fields.
How erasure works
Specific fields in the JSON-schema can be marked as erasable using an x-vendia-erasable definition.
Vendia will generate erase mutations in the GraphQL schema for all types containing erasable data.
Erase mutations can be executed against scalar and file objects to permanently erase data on either a node-local basis or across all nodes in the Uni.
For scalar data, erasure can be performed on a fine-grained basis down to the individual field level.
Erasure will erase data in 1) the current world state and 2) from all blocks in the ledger containing the field.
Once erased, fields cannot be reverse-engineered using any of the Vendia-generated hashes and cannot be "restored" by any node in the Uni.
Vaulted tokenization
Vendia uses a Vaulted Tokenization approach to support erasure while retaining the tamper-proof, verifiable, and immutable properties of the distributed ledger.
Mutations are stored on-chain in a tokenized form whereby erasable fields are replaced with cryptographically hashed values (tokens).
Plaintext field values are stored in a chain-adjacent vault. Queries to the ledger will automatically detokenize the mutations, replacing the tokens with the corresponding plaintext value for fields that have not been erased.
When erased, plaintext values are permanently and irrevocably removed from the vault but historical data in the ledger itself is never modified. Aside from erasure, values stored in the vault are never modified.
Transaction and block hashes are computed based on the tokenized form of the mutation containing cryptographic hashes of the erasable field values - thus the ledger remains immutable even if field are erased, and can be verified using the original tokenized form of the mutation.
Example of a tokenized mutation stored in the ledger:
mutation m {
addSelf_Person(id: "0187bd97-0593-cb3e-9c76-b71438598915",
input: {
firstName: "$${1ab347b85ee537bd26b1246f6cdde3d8}",
lastName: "$${2cb347b85ee537bd26b1246f6cdde34f}",
identification: {
type: "DRIVERS_LICENSE",
number: "$${a3h347b85ee537bd26b1246f6cdde3d8}"
}
}) {error}
}
Marking fields as erasable in the schema
The following data types at any level of the schema can be marked as erasable using the x-vendia-erasable definition:
- number
- integer
- string
- array
- boolean
Note that arrays can be marked as erasable but not elements within arrays. Nested fields within objects can be marked as erasable but not intermediate objects themselves.
Example:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "http://vendia.net/schemas/demos/basic_schema.json",
"type": "object",
"properties": {
"Person": {
"type": "array",
"items": {
"type": "object",
"properties": {
"identification": {
"type": "object",
"properties": {
"type": {
"type": "string"
},
"number": {
"type": "string"
},
"aliases": {
"type": "array",
"items": {
"type": "string"
}
}
},
"x-vendia-erasable": [
"number",
"aliases"
]
},
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
}
},
"x-vendia-erasable": [
"firstName",
"lastName"
]
}
}
}
}
Local vs. Global erasure
Local erasure erases the data on the local node but does not affect other nodes in the Uni. Once a field is erased locally, subsequent updates to the field originating on other nodes will be redacted (masked) on the local node.
Global erasure erases the data across all nodes in the Uni. Subsequent updates from any node will be redacted (masked) on all nodes.
By default, only the item owner can globally erase data. However, ERASE_GLOBAL ACL permissions can be granted to allow other nodes to globally erase an item.
Scalar data erasure
Fields can be erased from a given object using the generated erase mutation, i.e.
mutation erase {
erase_Person(
id: "0187bd97-0593-cb3e-9c76-b71438598915"
input: {firstName: GLOBAL, lastName: GLOBAL, identification: {number: GLOBAL} }
) {
result {
_id
}
}
}
Note that the erase mutation is ledgered as well as the status of asynchronous erasure operations.
File erasure
File data can be erased using eraseVendia_File mutation, i.e.
mutation erase {
eraseVendia_File(
id: "0187bd97-0593-cb3e-9c76-b71438598915"
eraseMode: LOCAL
syncMode: NODE_COMMITTED
) {
result {
_id
}
}
}
This will permanently erase all versions of the File from the CSP object store but will retain the file metadata in world state. The file metadata can be removed using remove_VendiaFile mutation.
Erasure status reporting
Erasure operations are executed asynchronously and their status is reported via EraseTask objects.
Erasure status is visible in the ledger via listVendia_BlockItems and can be queried directly using listVendia_EraseTaskItems and getVendia_EraseTask.
An erasure operation is guaranteed to be fully completed when the EraseTask status changes to SUCCESS.
i.e.
mutation m {
updateVendia_EraseTask(
id: "0187c2e5-2e10-341c-12fe-5acb9251bb6d",
input: {
completionTime: \"2023-04-27T13:26:53.645450791+00:00\",
status: SUCCESS
}
) {error}
}
Managing erasure permissions
All nodes have the ability to locally erase a field. Local erasure takes precedence over any configured ACL rules.
By default, only the item owner has permission to globally erase data. Global erasure permissions can be granted
to other nodes by setting ERASE_GLOBAL ACL permission on an object, i.e.
mutation grantGlobalErase {
update_Person(
id: "0187c2e4-b2b5-dafb-1f8e-1b29d062b0e1",
aclInput: {acl: [
{principal: {nodes: ["*"]}, operations: [READ, WRITE]},
{principal: {nodes: "PartnerNode"}, operations: [ERASE_GLOBAL]}
]
}
Masking
Once a field on an object has been erased, it can never be set again by further mutations to the object. The field will be masked in subsequent mutations and treated as redacted. For globally erased fields it will be masked on all nodes. For locally erased fields, it wil be masked on the nodes on which it has been locally erased.
Erasure metadata
Objects in world state contain metadata on field erasure state that can be inspected via existing get and list queries.
Additionally field hash and salt values can be inspected for verification purposes. For erased fields, the hash is retained but the salt is erased from the metadata.
i.e.
query getPersonMetadata {
get_Person(id: "0187c2e4-b2b5-dafb-1f8e-1b29d062b0e1") {
_meta {
erased {
nodes
type
}
fields {
firstName {
_erased {
nodes
type
}
_hash
_salt
}
identification {
number {
_erased {
nodes
type
}
_hash
_salt
}
aliases {
_erased {
nodes
type
}
_hash
_salt
}
}
lastName {
_erased {
nodes
type
}
_hash
_salt
}
}
}
firstName
lastName
identification {
aliases
number
type
}
}
}
Response:
{
"data": {
"get_Person": {
"_meta": {
"erased": null,
"fields": {
"firstName": {
"_erased": [
{
"nodes": [
"NodeTwo"
],
"type": "LOCAL"
}
],
"_hash": "e290549297653406a272eb24bd1a0ed6",
"_salt": null
},
"identification": {
"number": {
"_erased": [
{
"nodes": [
"NodeTwo"
],
"type": "LOCAL"
}
],
"_hash": "af654469baad9659ef1cf9ae40ad1950",
"_salt": null
},
"aliases": {
"_erased": [
{
"nodes": [
"NodeTwo"
],
"type": "LOCAL"
}
],
"_hash": "8905b53db042c04514b0e888e38a76d8",
"_salt": null
}
},
"lastName": {
"_erased": [
{
"nodes": [
"NodeTwo"
],
"type": "LOCAL"
}
],
"_hash": "4e9f52d53154487ebe2192ec16ff387c",
"_salt": null
}
}
},
"firstName": null,
"lastName": null,
"identification": {
"aliases": null,
"number": null,
"type": "DRIVERS_LICENSE"
}
}
}
}
Erasure vs. removal
Erasure erases the data permanently and irrevocably in the world state and the ledger, but does not remove the object metadata from the world state.
In some cases, it may be desirable to distinguish between erased and removed data. For example, erased objects contain metadata indicating the erasure state of the object which may be useful for tracking purposes.
A previously erased object can be removed using the corresponding remove mutation.
Verifying the ledger
Transaction and block hashes can be recomputed from ledger entries containing erasable/erased fields. To verify the ledger, the "tokenized"
form of the mutation must be used to compute the transaction hash. This can be selected in getVendia_Block and listVendia_Block queries using
the rawMutations selector.
Adding a new node
When a node is added to your uni, the new node will get different erasure settings depending on a few factors.
Globally Erased
If a field is globally erased, the field will also be erased in the new node
Locally Erased on the Seed Node
If a field is locally erased on the Seed Node, the field will be not copied to the new node. However, the new node will get any updates on the field going forward
Locally Erased on the non-seed Node
If a field is locally erased on a non-seed node, the field will be backfilled for the new node. The new node will also continue to get any updates on the field