Contract Keys in Canton

DAML provides a “contract key” mechanism for contracts, similar to primary keys in relational databases. Canton supports the full syntax of contract keys, but their full semantics is provided only in specialized setups. That is, all valid DAML contracts using keys will run on Canton, but their behavior may deviate from the prescribed one. This document explains the deviation, as well as ways of recovering the full functionality of keys in some scenarios. It assumes a reasonable familiarity with DAML.

Keys have two main functions:

  1. Simplifying the modeling of mutable state in DAML. DAML contracts are immutable and can be only created and archived. Mutating a contract C is modeled by archiving C and creating a new contract C' which is a modified version of C. Other than keys, DAML offers no means to capture the relation between C and C'. After archiving C, any contract D that contains the contract ID of C is left with a dangling reference. This makes it cumbersome to model mutable state that is split across multiple contracts. Keys provide mutable references in DAML; giving C and C' the same key K allows D to store K as a reference that will start pointing to C' after archiving C.
  2. Checking that no active contract with a given key exists at some point in time. This mainly serves to provide uniqueness guarantees, which are useful in many cases. One is that they can serve to de-duplicate data coming from external sources. Another one is that they allow “natural” mutable references, e.g., referring to a user by their username or e-mail.

A Canton domain can be run in two modes:

  1. In normal mode, contract keys in Canton provide the first, but not the second function, at least not without additional effort or restrictions. In particular:
    1. In Canton, two (or more) active contracts with the same key may exist simultaneously on the same or different domains.
    2. A lookupByKey @Template k may return None even when an active instance of template Template with the key k exists.
    3. A fetchByKey @Template k or an exerciseByKey @Template k or a positive lookupByKey @Template k (returning Some cid) may return any active contract of template Template with key k.
  2. In unique-contract-key (UCK) mode, contract keys in Canton provide both functions; there can be at most one active contract for each key on a UCK domain. However, participants can connect to at most one UCK domain and this UCK domain is the only domain that the participant can connect to. UCK domains and their participants are thus isolated islands that are deprived of Canton’s composability and interoperability features.

In the remainder of the document we:

  1. give more detailed examples of the differences above
  2. give an overview of how keys are implemented so that you can better understand their behavior
  3. show workarounds for recovering the uniqueness functionality in particular scenarios on normal domains
  4. give a formal semantics of keys in Canton, in terms of the DAML ledger model
  5. explain how to run a domain in UCK mode.

Normal mode

This section explains how contract keys behave on participants connected to normal Canton domains.

Examples of Semantic Differences

Double Key Creation

Consider the following template:

template Keyed
  with 
    sig: Party
    k: Int
  where
    signatory sig
    key (sig, k): (Party, Int)
    maintainer key._1

The DAML contract key semantics prescribes that no two active Keyed contracts with the same keys should exist. This is exemplified in the following DAML scenario:

multiple = scenario do
  alice <- getParty "alice"
  submitMustFail alice $ do
    create $ Keyed with sig = alice, k = 1
    create $ Keyed with sig = alice, k = 1

Alice’s submission must fail, since it attempts to create two contracts with the key (Alice, 1). In Canton, however, the submission is legal, and will succeed (if executed, for example, through DAML Script). Thus, you cannot directly rely on keys to ensure the uniqueness of user-chosen usernames or external identifiers (e.g., order identifiers, health record identifiers, entity identifiers, etc.) in Canton.

False lookupByKey Negatives

Similarly, your code might rely on the negative case of a lookupByKey, for example as follows.

template Initialization
  with
    sig: Party
    k: Int
  where
    signatory sig

template Orchestrator
  with
    sig: Party
  where
    signatory sig

    controller sig can 
      nonconsuming Initialize: Optional (ContractId Initialization)
        with
          k: Int
        do
          optCid <- lookupByKey @Keyed (sig, k)
          case optCid of
            None -> do
              create Keyed with ..
              time <- getTime
              cid <- create Initialization with sig, k
              pure $ Some cid
            Some _ -> pure None

If you run some process (represented by the Initialization template here), you may usually use a pattern like above to ensure that it is run only once. The Initialization template does not have a key. Nevertheless, if all processing happens through the Orchestrator template, there will only ever be one Initialization created for the given party and key. For example, the following scenario creates only one Initialization contract:

lookupNone = scenario do
  alice <- getParty "alice"
  orchestratorId <- submit alice do
    create Orchestrator with sig = alice
  submit alice do 
    exercise orchestratorId Initialize with k = 1
  submit alice do 
    exercise orchestratorId Initialize with k = 1

In scenarios, transactions are executed sequentially. Alice’s second submission above will always find the existing Keyed contract, and thus execute the Some branch of the Initialize choice. In real-world applications, transactions may run concurrently. Assume that initTx1 and initTx2 are run concurrently, and that these are the first two transactions running the Initialize choice. Then, during their preparation, both of them might execute the None branch (i.e., lookupByKey might return a negative result), and thus both might try to create the Initialization contract. However, negative lookupByKey results must be committed to the ledger, and the key consistency requirements prohibit both of them committing. Thus, one of initTx1 and initTx2 might fail, or they both might succeed (if one of them sees the effects of the other and then executes the Some branch), but in either case, only one Initialization contract will be created.

In Canton, however, it is possible that both initTx1 and initTx2 execute the None branch, yet both get committed. Thus, two Initialization contracts may get created.

Semantics of fetchByKey and Positive lookupByKey

DAML also provides a fetchByKey operation. DAML commands are evaluated against some active contract set. When DAML encounters a fetchByKey command, it tries to find an active contract with the given key (and fails if it cannot). Since DAML semantics prescribe that only one such contract may exist, it is clear which one to return. For example, consider the scenario:

fetchSome = scenario do
  alice <- getParty "alice"
  keyedId1 <- submit alice do
    create Keyed with sig = alice, k = 1
  keyedId2 <- submitMustFail alice do
    create Keyed with sig = alice, k = 1
  (foundId, _) <- submit alice do 
    fetchByKey @Keyed (alice, 1)
  assert $ foundId == keyedId1
  optFoundId <- submit alice do 
    lookupByKey @Keyed (alice, 1)
  assert $ optFoundId == Some keyedId1

DAML’s contract key semantics says that Alice’s second submission must fail, since a contract with the given key already exists. Thus, her third submission will always succeed, and return keyedId1, since this is the only Keyed contract with the key (Alice, 1). Similarly, her fourth submission will also successfully find a contract, which will be keyedId1.

As discussed earlier, Alice’s second submission in the above scenario will however succeed in Canton. Alice’s third and fourth submissions thus may return different contract IDs, with each returning either keyedId1, or keyedId2. Whichever one is returned, a successful fetchByKey and lookupByKey still guarantees that the returned contract is active at the time when the transaction gets committed. As mentioned earlier, negative lookupByKey results may be spurious.

Canton’s Implementation of Keys

Internally, a Canton participant node has a component that provides the gRPC interface (the “Ledger API Server”), and another component that synchronizes participants (the “sync service”). When a command is submitted, the Ledger API Server evaluates the command against its local view, including the resolution of key lookups (lookupByKey and fetchByKey). Submitted commands are evaluated in parallel, both on a single node, as well as across different nodes.

The evaluated command is then sent to the sync service, which runs Canton’s commit protocol. The protocol provides a linear ordering of all transactions on a single domain, and participants check all transactions for conflicts, with an earlier-transaction-wins policy. As participants only see parts of transactions (the joint projection of the parties they host), they only check conflicts on contracts for which they host stakeholders. During conflict detection, positive key lookups (that find a contract ID based on a key) are treated as ordinary fetch commands on the found contract ID, and the contract ID is checked to still be active. Negative key lookups, on the other hand, are never checked by Canton (a malicious submitter, for example, can always successfully claim that the lookup was negative). Similarly, contract creations are not checked for duplicate keys. Logically, both of these checks would require checking a “there is no such key” statement. Canton does not check such statements. While adding the check to the individual participants is straightforward, it’s hard to get meaningful guarantees from such local checks because each participant has only a limited view of the entire virtual global ledger. For example, the check could pass locally on a participant even though there exists a contract with the given key on some domain that the participant is not connected to. Similarly, since the processing of different domains runs in parallel, it’s unclear how to consistently handle the case where transactions on different domains create two contracts with the same key.

For integrity, the participants also re-evaluate the submitted command (or, more precisely, the subtransaction in the joint projection of the parties they host). The commit protocol ensures that any two involved participants will evaluate the key lookups in the same way as the Ledger API Server of the submitting participant. That is, if there are two active contracts with the key k, the protocol insures that a fetchByKey k will return the same contract on all participants.

Once the sync protocol commits a transaction, it informs the Ledger API server, which then atomically updates its set of active contracts. The transactions are passed to the Ledger API server in the order in which they are recorded on the ledger.

Workarounds for Recovering Uniqueness

Since achieving some form of uniqueness for ledger data is necessary in many cases, we list some strategies to achieve it in Canton without being locked into a UCK domain. The strategies’ applicability depends on your contracts and the deployment setup of your application. In general, none of the strategies apply to the case where creations and deletions of contracts with keys are delegated.

Setting: Single Maintainer, Single Participant Node

Often, contracts may have a single maintainer (e.g., an “operator” that wants to have unique user names for its users). In the simplest case, the maintainer party will be hosted on just one participant node. This setting allows some simple options for recovering uniqueness.

Command ID Deduplication

The Ledger API server deduplicates commands based on their IDs. Note, however, that the IDs are deduplicated only within a configured window of time. This can simplify the uniqueness bookkeeping of your application as follows. Before your application sends a command that creates a contract with the key k, it should first check that no contract with the key k exists in a recent ACS snapshot (obtained from the Ledger API). Then, it should use a command ID that’s a deterministic function of k to send the command. This protects you from the race condition of creating the key twice concurrently, without having to keep track of commands in flight. Caveats to keep in mind are:

  • you need to know exactly which contracts with keys each of your commands will create
  • your commands may only create contracts with a single key k
  • only the maintainer party may submit commands that create contracts with keys (i.e., do not delegate the creation to other parties).

However, these conditions are often true in simple cases (e.g., commands that create new users).

Generator Contract

Another approach is to funnel all creations of the keyed contracts through a “generator” contract. An example generator for the Keyed template is shown below.

template Generator
  with
    sig: Party
  where
    signatory sig

    controller sig can
      Generate : (ContractId Generator, ContractId Keyed)
        with
          k: Int
        do
          existing <- lookupByKey @Keyed (sig, k)
          keyed <- case existing of
            Some cid -> pure cid
            None -> 
              create Keyed with ..
          gen <- create this
          pure (gen, keyed)

The main difference to the Orchestrator contract is that the Generate choice is consuming. Caveats to keep in mind are:

  • Your application must ensure that you only ever create one Generator contract (e.g., by creating one when initializing the application for the first time).
  • All commands that create the Keyed contract must be issued by the maintainer (in particular, do not delegate choices on the Generator contract to other parties).
  • You must not create Keyed contracts by any other means other than exercising the Generate choice.
  • The Generate choice above will not abort the command if the contract with the given key already exists, it will just return the existing contract. However, this is easy to change.
  • This approach relies on a particular internal behavior of Canton (as discussed below). While we don’t expect the behavior to change, we do not currently make strong guarantees that it will not change.
  • If the participant is connected to multiple domains, the approach may fail in future versions of Canton. To be future proof, you should only use it in the settings when your participant is connected to a single domain.

A usage example scenario is below.

generator = scenario do
  alice <- getParty "Alice"
  -- Your application must ensure that the following command runs at most once
  gen <- submit alice $
    create Generator with sig = alice
  (gen, keyed) <- submit alice $
    exercise gen Generate with k = 1
  (gen, keyed1) <- submit alice $
    exercise gen Generate with k = 1
  assert $ keyed1 == keyed
  submit alice $
    exercise keyed Archive
  (gen, keyed2) <- submit alice $
    exercise gen Generate with k = 1
  assert $ keyed2 /= keyed

To understand why this works, first read how keys are implemented in Canton. With this in mind, since the Generate choice is consuming, if you issue two or more concurrent commands that use the Generate choice, at most one of them will succeed (as the Generator contract will be archived by the time the first transaction commits). Thus, all accepted commands will be evaluated sequentially by the Ledger API server. As the server writes the results of accepted commands to its database atomically, the Keyed contract created by one command that uses Generate will either be visible to the following command that uses Generate, or it will have been archived by some other, unrelated command in between.

Setting: Single Maintainer, Multiple Participants

Ensuring uniqueness with multiple participants is more complicated, and adds more restrictions on how you operate on the contract.

The main approach is to track all “allocations” and “deallocations” of a key through a helper contract.

template KeyState
  with
    sig: Party
    k: Int
    allocated: Bool
  where
    signatory sig

    controller sig can
      Allocate : (ContractId KeyState, ContractId Keyed)
        do
          assert $ not allocated
          newState <- create this with allocated = True
          keyed <- create Keyed with ..
          pure (newState, keyed)

      Deallocate : ContractId KeyState
        do
          assert $ allocated
          (cid, _) <- fetchByKey @Keyed (sig, k)
          exercise cid Archive
          create this with allocated = False

Caveats:

  • Before creating a contract with the key k for the first time, your application must create the matching KeyState contract with allocated set to False. Such a contract must be created at most once. Most likely, you will want to choose a “master” participant on which you create such contracts.
  • Do not delegate choices on the Keyed contract to parties other than the maintainers.
  • You must never send a command that creates or archives the Keyed contract directly. Instead, you must use the Allocate and Deallocate choices on the KeyState contract. The only exception are consuming choices on the Keyed contract that immediately recreate a Keyed contract with the same key. These choices may also be delegated.

A usage example scenario is below.

state = scenario do
  alice <- getParty "Alice"
  -- Your application must ensure that the following command executes at most once
  state <- submit alice $
    create KeyState with sig = alice, k = 1, allocated = False
  (state, keyed) <- submit alice $
    exercise state Allocate
  submitMustFail alice $
    exercise state Allocate
  -- If you archive the keyed contract without going through the
  -- KeyState, you must also recreate it in the same transaction.
  -- For example, if Keyed had consuming choices, the choices' bodies
  -- would have to recreate another Keyed contract with the same key
  submit alice $ do
    exercise keyed Archive
    create Keyed with sig = alice, k = 1
  state <- submit alice $
    exercise state Deallocate
  (state, keyed2) <- submit alice $
    exercise state Allocate
  assert $ keyed2 /= keyed

An alternative to this approach, if you want to use a consuming choice ch on the Keyed template that doesn’t recreate key, is to record the contract ID of the KeyState contract in the Keyed contract. You can then call Deallocate from ch, but you must first modify Deallocate to not perform a lookupByKey.

Setting: Multiple Maintainers

Achieving uniqueness for contracts with multiple maintainers is more difficult, and the maintainers must trust each other. To handle this case, follow the KeyState approach from the previous section. The main difference is that the KeyState contracts must have multiple signatories. Thus you must follow the usual DAML pattern of collecting signatories. Beware that you must still structure this such that you only ever create one KeyState contract.

Formal Semantics of Keys in Canton

In terms of the DAML ledger model, Canton’s virtual shared ledger satisfies key consistency only when it represents a single UCK domain. In general, Canton’s virtual shared ledger violates key consistency. That is, NoSuchKey k actions may happen on the ledger even when there exists an active contract with the key k. Similarly, Create actions for a contract with the key k may appear on the ledger even if another active contract with the key k exists.

In terms of DAML evaluation, i.e., the translation of DAML into the ledger model transactions, the following changes:

  • When evaluated against an active contract set, a fetchByKey k may result in a Fetch c action for any active contract c with the key k (in Canton, there can be multiple such contracts). In the current implementation, it will favor the most recently created contract within the single transaction. However, this is not guaranteed to hold in future versions of Canton. If no contract with key k is active, it will fail as usual.
  • Similarly, lookupByKey k may result in a Fetch c for any active contract c with the key k of which the submitter is a stakeholder. If no such contract exists, it results in a NoSuchKey k as usual.
  • Likewise, an exerciseByKey k may result in an Exercise on any contract c with the key k. It fails if no contract with key k is active.

Domains with uniqueness guarantees

A Canton domain can be configured to provide unique contract key (UCK) semantics. The semantic differences of normal domains disappear if the transactions are submitted to a participant connected to a Canton domain in UCK mode. The workarounds are not needed then. The UCK mode can be configured in the domain configuration file by setting the unique-contract-key flag.

canton {
  domains {
    unique-contract-key-domain {
      domain-parameters.unique-contract-keys = true

      storage = ${_shared.storage}
      storage.database-name = "uck-domain"
      public-api.port = 10018
      admin-api.port = 10019
    }
  }
}

Participants can connect to a UCK domain only if they have never connected to a different domain. Moreover, once they have successfully connected to a UCK domain, they will refuse to connect to other domains. Accordingly, conflict detection on a single domain suffices to check for key uniqueness. Participants connected to a UCK domain check for key conflicts whenever they host one of the key maintainers:

  • When a contract is created, they check that there is no other active contract with the same key.
  • When the submitted transaction contains a negative key lookup, the participants check that there is indeed no active contract for the given key.

Warning

DAML workflows deployed on a UCK domain are locked into this domain. They cannot use in Canton’s composability and interoperability features because the participants will refuse to connect to other domains.