Open Discussion: Umbrella Arbitration Criteria

Umbrella Arbitration Criteria:

When considering our role as arbiters within the Umbrella system, we must carefully lay out how we plan to determine the validity of an exploit. The two main protection providers today (Cover and Nexus) take highly divergent paths. Cover has no publicly stated criteria and the community is free to vote as they please, before it goes to a council of auditors that make a final ruling. Nexus, on the other hand, requires proof of loss when submitting a claim, as well as requiring the hack to specifically occur in the covered protocol (ie. Harvest exploit resulting from Curve manipulation would not be covered for Harvest protection seekers).

There are multiple angles to consider here:

  • Arb vs. Exploit - Code is Law?: Where is the line for an exploit vs. intended code functionality that was not fully appreciated by users? What’s an arb vs. an exploit? A good example is Compound’s recent DAI liquidations as a result of single-source Coinbase oracle. It was always known that Coinbase is the single oracle, and this was a somewhat known attack vector – is it an exploit?

    • Proposed Policy: When you get car insurance or home insurance, it’s because you know you might get in a wreck or have a home fire, but it is not expected or intended behavior. In general, Yam’s guiding principle should be related to loss of funds related to unexpected or unintended behavior.
  • Composability vector: Affected protocol is NOT the manipulated protocol (ie. all the Curve flashloan exploits) - Which does Umbrella protect?

    • Proposed Policy: Umbrella is most concerned with loss of funds. In most of the Curve cases, Curve LPs actually profited. Umbrella will arbitrate claims according to loss of funds – if a composability exploit results in loss of funds for more than one protocol, each would be valid for a payout.
  • Severity of Exploit: Sushi recently experienced an exploit totalling approximately $15k in lost rewards prior to the exploit being discovered and stopped. Cover did not consider this a valid claim. What is Yam’s threshold for a valid exploit that deserves a payout? This is likely the most difficult decision to make.

    • Proposed Policy: Perhaps there is a threshold that exists as a ratio of (exploit lost funds) / (total coverage purchased) such that if the amount lost to exploit is less than say 50% of coverage purchased, a payout does not occur.

This is an open discussion, and there may be additional considerations to account for and different solutions.

5 Likes

Arb vs Exploit: Your line of reasoning works well. The “unexpected” or “unintended behavior” is a good principle.

Everything that’s known is the underwriter’s risk exposure. For instance: If someone provides coverage for something that says “…protocol functions with a single oracle (namely Uniswap)” then this is a known fact and attack vector. Assessing the likelihood of such exploit is up to the underwriter.

Composability: In general, this sounds good. I cannot remember the details of the Harvest exploit.

Is there a limit to composabilty? Insofar as layers of composability would encourage moral hazard by the protection buyer.

Severity: Normally it’s the other way around. Small claims are easily disbursed, big ones are challenged. Small claims might be relevant to smaller investors who are reliant on the insurance. Also, “paying out” is the core function of an insurance. A track record of payouts should instill confidence by insurance buyers in Umbrella.

I’d argue it’s the other way around: There’s a threshold above which arbitration becomes relevant. Below that, it’s a less critical arbitration discussion.

Also, what’s the likelihood of small exploits happening? Most hackers don’t take the risk for $20k. Smaller damages might be more genuine bugs vs the really big ones.

There’s also the possibility that a policy only kicks in above a certain threshold. That would also work with what we have seen in DeFi: <$100k exploits/bugs get reimbursed by the protocol (eg Hegic afaik). It’s the big damages they want to insure against.

3 Likes

我认为应该对购买保险者有一些操作规范或是法律层面的约束,其他标准就好办了

Hey great thoughts @trente + @nonstopTheo

Arb vs Exploit

Ya “unexpected” or “unintended behavior” is a great principle.

We can use the parameters set by the MetaPool Designers to help decide what falls under “unexpected” / “unintended.”

(One MetaPool Designer, for example, may clearly state liquidations related to single-source oracles are not “unexpected” and therefore not an exploit.)

Composability

100% def, I like the mindset here a lot trent. Would want to see composability and rule where loss of funds occurred.

Severity of Exploit

I agree with Theo - we want to payout, it instills confidence in Umbrella. I support a percentage payout (if it falls under the ratio of (explot lost funds)/(total coverage purchased) vs all or nothing.

awesome thoughts! thanks for kickin this conversation off trent

1 Like

Due to the lack of understanding in the insurance domain, some general thoughts:

  • Would it be possible to create some kind of market research about the most epic defi fails that fit into the categories described by @trente above? I’m think about some kind of one-pagers/summary per fail to explain it to a six year old :stuck_out_tongue_winking_eye:
  • Next step would be to assign/map these incidents to the relevant categories
  • One option would to start with a phased approach and to beging to provide coverage for incidents with the most simple criteria to determine the validity of the exploit. This approach does only makes sense if there is an overlap between strong market demand and easy to settle claims/incidents.
  • If we would think of an auditor council, would make sense to try to engage relevant white hat hackers for participating in the council?

One last thing, how do we mitigate this risk:

  1. Create a backdoor which looks like an accidental exploit
  2. Utilize the backdoor to hack the protocol
  3. Settle the insurance claim on top of exploiting the protocol.
  4. Sometime even the funds are returned after hacks, but the token price is -90%. Would the see the token price <90% as a loos that is covered by an insurance?
2 Likes

We can certainly create a list of DeFi hacks and examples of how YAM would vote based on the criteria we develop. Ultimately people are going to be able to vote however they want, however, and we need to be cautious of that.

Regarding a phased approach to what we cover, it’s not a bad idea but I do think we need to cover most known exploits and anything that is just a straight up logic bug (like Cover).

Auditor council is interesting. I could also see teaming up with Kleros. Could make the arbiter a 2/3 multisig: Yam, Kleros, Council, to grant a payout, though that’s a lot of overhead.

No real way around insurance on top of being the exploiter at this point :confused:

1 Like

Regarding Severity:

I’ve asked Brock to create the ability for partial payouts. The best way to calculate this imo would be to pay out a percentage (funds lost) / (TVL in contract or protocol). I think this is a huge improvement for protection providers, as a 100k hack on 1M of protection purchased will no longer result in a 1M payout. The other way to do it would be (funds lost) / (protection purchased).

How we perform this calculation can also be a discussion, from a code perspective all that’s implemented is the ability to payout a percentage of coverage. It can also depend on what’s being protected (specific contract vs. entire protocol).

3 Likes

Collaborating with Kleros as Arbiters is an idea I’m interested in. I feel like it could help bring a lot of trust and confidence to Umbrella.

I can imagine some scenarios where it would be beneficial to bring an organization like Kleros on board:

  • When a claim is denied, we could have an appeals process that’s sent to Kleros for a second opinion.

  • If the Yam community can’t come to a consensus, or fails to reach quorum on a vote.

  • If there’s a conflict of interest, like let’s say Yam Treasury is the main Protection Provider.

2 Likes

In addition, will there some kind of formal / technical prerequisites to get accepted for protection or to determine the insurance rates?

Insurance rate = There more of these prerequisites are met, the lesser risk and payment to insurance(Simpilfied)

For example:

  • Minimum one / multiple audit reports
  • Non anon team
  • Pen testing by additional hired security experts
  • Mandatory bug bounty program
  • Development infrastructure:
    • CI/CD pipeline
    • Automated code quality checks
    • AI checks to finds bugs
    • Frequent review of GitHub project quality metrics
  • If applicable: Review of DAO setup and governance processes (Multisig / Onchain votes)

Umbrella Terms and Conditions - Questions and Concerns

I have been working through calibrating the terms and conditions, which include arbitration criteria, and there are some issues that need to be resolved. In the quotes below I am paraphrasing the original statements from @trente and adding in the information discussed in the rest of the thread. It is a long one, so grab your favorite beverage and dive in with me.

#1

Arb vs. Exploit - Code is Law?:

Where is the line for an exploit (unintended code functionality that was not expected by the developers or users) vs. intended code functionality that was not fully appreciated by users?

What’s an arb vs. an exploit? A good example is Compound’s recent DAI liquidations as a result of single-source Coinbase oracle. It was always known that Coinbase is the single oracle, and this was a somewhat known attack vector – is it an exploit?

Proposed Policy:

When you get car insurance or home insurance, it’s because you know you might get in a wreck or have a home fire, but it is not expected or intended behavior.

In general, Yam’s guiding principle should be related to loss of funds related to unexpected or unintended behavior.

This seems to be generally agreed upon for a starting philosophy of how we handle arbitrations. Starting off with the low hanging fruit.

#2

Composability:

If the affected protocol is NOT the manipulated protocol, (ie. all the Curve flashloan exploits) which does Umbrella protect?

Proposed Policy:

Umbrella is most concerned with loss of funds. In most of the Curve cases, Curve LPs actually profited. Umbrella will arbitrate claims according to loss of funds – if a composability exploit results in loss of funds for more than one protocol, each would be valid for a payout.

Open Question: Do we need to distinguish between exploits in the code vs social attacks? What about rug pulls?

In the event of a rug pull, the people in the pool clearly have lost money, but the protocol has functioned exactly as intended. Someone who owned tokens in the AMM pool lost money because they deposited into the AMM pool, even if the AMM pool itself was not exploited. The “exploit” happened elsewhere and took advantage of the mechanism of the AMM to function. If covering a protocol like Uniswap, the terms and conditions would need to explicitly exclude rug pulls, or exclude non-code related exploits entirely.

But have we now invalidated our answer to question 1, where we want to cover losses that occur due to unexpected behavior. Should the users of an unsafe project have expected a Rug pull?

Can this be avoided by making it very clear as to what the “correct and intended” function of a protocol is in our terms and conditions? In the example above, Uniswap functioned exactly as intended, so there would be no payout.

But, is there an edge case where someone comes up with a new exploit and takes advantage of the way Uniswap is supposed to work to drain funds or extract value? Do we pay out for that?

There is clear grey area here about what “functioning correctly” means that the DAO (or other arbiter) will have to deliberate about explicitly and in an educated way to make a fair decision in a claim event. Setting a precedent one way could impact the perceived trustworthiness of the protocol for the foreseeable future.

#3

Severity of Exploit:

Sushi recently experienced an exploit totaling approximately $15k in lost rewards prior to the exploit being discovered and stopped. Cover did not consider this a valid claim. What is Yam’s
threshold for a valid exploit that deserves a payout?

Proposed Policy:

Partial Payouts will be calculated as funds lost in exploit / TVL in exploited contract or protocol. So an exploit that loses $100k from a contract that held $1M of funds will result in a payout of 10% of the purchased coverage.

Open Question/Concern:
How do we allow users to buy protect for specific pools within products while still allowing protection providers to cover many products and deal with changes to the products that are covered.

Diving into this question requires a bit of background on how Umbrella works at a high level:

In Umbrella as it is designed, there is no way to ascertain actual loss suffered by any specific protection buyer in an exploit. The contract pays out based on whether certain criteria (an exploit) happened and the protocol does not care if the person who is paid actually lost funds, as long as someone did. The person who is paid is just the person who bought protection. So it functions more like a prediction market with the DAO as an oracle than it does a normal insurance market.

Concepts are the name for what the Umbrella Smart Contract will store as a specific product/protocol/pool/etc. to be covered. They can be settled individually and are the individual “pools” that protection seekers buy. A concept is just something that a protection seeker can buy protection on and the arbiter will need to make a decision upon if there is an exploit. It can have any level of specificity we desire.

Broad or Narrow Concepts?

Up until now we have talked about defining concepts pretty broadly (i.e. uniswap, or ledger hardware wallets). This allows a lot of flexibility because the defining your concept broadly allows it to change without needing to re-make the contract. If I am providing protection for “Uniswap” then it doesnt matter if new pools are added or created because they still fall under the concept.

But there is a tradeoff here for protection buyers. It does not seem possible to allow users to buy protection for specific parts of a protocol that they use and also provide this broad flexibility. let me use an example with a concept that covers all Yearn vaults.

Let’s say we create a metapool with a concept that covers Yearn as a whole and the DAI vault is exploited for 10% of its TVL. As of this writing, the DAI vault has a TVL of 500M while Yearn has a TVL of 2.6B as a whole. That means that the DAI vault TVL is ~20% of the total TVL. If I had funds in the DAI vault, I would expect to be paid out 20% of my protection purchased to be made whole. But this could only work if I bought coverage for the DAI vault specifically. If the coverage that I bought is for Yearn as a whole, then anyone who bought that same protection on Yearn should get a payout that is equal to the $-value lost in the exploit compared to the total Yearn TVL (10% * 20% = 2%). Anyone who bought protection for Yearn (even if they bought protection for another Vault that was not affected) would receive the same payout as me (2%). If I bought protection thinking that I would be fully protected in the event of an exploit of the DAI vault, then got screwed since I lost 20% of my “protected” funds and only got paid out 2%.

This breaks the social contract of “protection” and turns Umbrella back into a betting market on whether a protocol will get hacked. If anything, this would be more like insurance for token holders who may now lose money through token depreciation due to a hack. Interesting, but not what we have been discussing as the function for Umbrella.

A Potential Fix

To make this work as I believe we intend it to, and to correctly calibrate the magnitudes and payouts of exploits, we would would need to build our concepts so that they are granular to a level in which protection buyers can specify which products they want to cover. If this granularity is developed to a level where pools are not divisible into smaller ones, The arbiter will be able to accurately determine who to pay out and buyers can be confident in getting an accurate payout based on what they bought.

Again Let’s take Yearn as an example. A metapool could cover all of the Yearn Vaults and individual protection buyers would buy individual vault protection (the concepts). So you would have concepts for the “DAI vault” and the “3CRV Vault” and so on. Now we have a clear way to fairly determine a payout for someone who bought coverage on a vault that was hacked. The TVL is not further divisible and only those who own this concept can claim in an exploit.

Tradeoffs!

This seems like a good solution, but there are trade-off here as well. This idea works as long as the number of concepts that can be purchased remains mostly consistent over time, or if the concepts can be updated. If concepts are something that are explicitly defined in the creation of the smart contract (as it currently works), then the only way to add or remove concepts would be to create a new metapool and migrate to that contract. This is not something that is possible in an automatic way with the current contract. It could be done manually by the pool participants, but this has a number of issues and open questions:

  • Protection providers are limited in how much they can withdraw from the contract. This is determined by the utilization rate. So they could not all migrate in a coordinated way without protection buyers also migrating.
  • Protection buyers have purchased protection at a specific rate on the utilization curve. Migrating to a new contract would mean that they will probably pay a new rate. This could be better or worse for the buyer, but either way it does not make the transition simple.
  • Ideally, you would promise lower rates on the new contract by moving funds there, but this is difficult or impossible if your funds are locked up in the existing contract. Without funds in the new contract, existing buyers may choose not to migrate, or there may not be enough liquidity.
  • Incentives or a dedicated pool of liquidity to facilitate migrations could help this problem.
  • A migration tool/event that would queue existing buyers and providers into the new pool would be an interesting, but potentially very complicated solution.

This will define how Umbrella works

If we want Umbrella to be “protection” for users who want to safeguard their funds deposited in a DeFi protocol then we will need to cover the specific pools of funds that could get hacked. This is possible but without upgradability of the contracts this may be hard to adjust as projects add features or pools.

Our other option is to accept the current limitations with broad concepts and be clear that Umbrella functions as a high level bet for or against a specific protocol or product being exploited, and not always as specific protection for funds that you may deposit in that protocol.

How we do this will impact who will want to use our product, how we market it, and how the user flow and website interfaces work. I am curious to hear others’ thoughts on these issues and whether there is an optimal path forward.

What if we don’t do partial payments?

As an addendum, if anyone is wondering if not doing partial payments helps solves our problem, the answer is no, and here is why:

With no partial payments, an exploit will trigger the pay-out of the full amount of coverage that protection seekers have bought. This “solves” the problem of a protection buyer not getting their fair share of a payout in the event of an exploit, but makes the system significantly more risky to protection providers who are now on the hook for large payouts in the event of a minor hack.

If we keep this mechanism as is, then it would make sense for protection providers to demand higher premiums to offset the risk that they have now taken on. Higher premiums might be acceptable for a user who is looking to speculate on whether a protocol gets hacked, but is not efficient for a user who wants to protect their funds in one specific part of a protocol. Even if we add granularity to the contract to allow people to buy indivisible concepts where the arbiter can accurately determine a payout, they are now paying for additional protection that they don’t need since they will potentially get paid out more than they lost in the event of an exploit.

We could try to lower the premiums that protection buyers have to pay by setting a threshold for a payout. A threshold for payout would prevent small claims from being paid out and lower the costs for protection providers, but at a large UX cost for buyers. This would move the model to one similar to American health insurance where the threshold percentage acts like a deductable that must be paid first by the buyer.

My experience is that most Americans hate having to deal with deductibles and out of pocket expenses but only do so because they have no other option due to availability or cost. While this may help attract some users to the product, it seems like a sub-optimal design decision over trying to incorporate partial payments.

RE: #2 Rug Pulls

I would generally consider this to be outside of intended/expected behavior on behalf of the user. They were likely unaware that such an exploit were possible –– in coverage of a specific Uniswap pool that gets rugpulled, imo this should be covered.

Which brings us to #3 and specificity. We should definitely NEVER cover “Uniswap” – it is far far too vast.

The original conceit of the protocol was always to have more specific coverage – down to the specific contract level. Not only does this have a number of advantages for those purchasing protection, it also is a major point of differentiation vs. the other protection solutions on the market. The degree to which the system is built towards this end is clear in the issues you raise regarding proportional payouts.

So you if wanted cover Yearn, the best way would be to set up a Yearn Metapool that includes specific pools for each vault, as well as a “Future Vault” pool, such that any new vaults created would be covered by that pool. It’s kind of hack-y but it does work.

In a V2 of this project that could function more similarly to Bento or Balancer V2, this process can be a little easier – but it currently too gas-intensive for L1 and the V2 code has not been developed.

In general, partial payment do seem necessarily in almost all scenarios to allow protection providers to feel secure. Otherwise they have huge risk, which would demand huge premiums, which would likely eliminate purchase demand.

2 Likes

Hey Ross, regarding partial payments:

While I agree that we need partial payouts, I have some questions on the partial payouts formula (funds lost in exploit / TVL in exploited contract or protocol). Essentially, how to determine what is the TVL in the exploited contract. Do we take a snapshot before the hack (what if there are multiple hacks reasonably spaced out, where do you take the snapshot?). NXM seems to be tackling this via the voting process, i.e. arbiters vote for how much the claim should be with some sort of averaging process. See original post by Hugh.)

Interesting. In addition, if Yearn ever comes out with some degen product, Yam can potentially create a separate metapool providing coverage to the said degen products.

ok, I follow this logic. This works when combined with the idea of “narrow” concepts where a specific pool is covered and not the protocol as a whole. The money in that pool was exploited so they would get a payout.

I agree with this. But it changes how we have been talking about some of the metapools that we are looking to make. The Bluechips metapool that we discussed here proposed protecting MakerDAO, Compound, and Uniswap. We will have to change this to remove uniswap, and any other protocols that have unbounded numbers of internal pools (or specify exactly which pools are covered). We could cover the biggest 10 pools and create concepts for each of those.

The problem I see with a single “Future Vault” pool is that once there is more than one pool in this Future Vault then you start running into the issue of not being able to determine who should get paid out in the event of an exploit. But you could instead create multiple unused concepts that are not initially available through the UI, that then get mapped to a specific new vault as they become available. The expandability of the metapool would be limited by the number of unused concepts available, and would need to be migrated when all the concepts are used, or when we are close to using them all. I am curious how this would work in practice and how to map the concepts to the new vaults after the creation of the metapool.

This determination is made by the arbiter and defined by the terms and conditions of the contract. So when an exploit is reported, the job of the arbiter (in our beta version this will be the YAM DAO) is to determine the validity of the exploit, to determine what block or blocks it occurred at, and to determine how much was lost and how much was in the pool at the block prior to the exploit.

I have not considered attacks that are spaced out enough to make it unclear whether it was 1 hack or multiple, but as far as I can tell this doesn’t happen very often. Most attacks occur in a relatively small period of time and then a protocol is either drained, frozen, or fixed.

There is a period where funds in the protection contract are locked while arbitration happens, so it would depend on whether the second exploit happened in that window or not. If there were a second hack, it could get reported and have a concurrent, second arbitration event that would lock the pool for an additional period of time to sort out.

But we would need to consider whether the original snapshot for TVL should be used or a new snapshot at the block of the new exploit would dictate the payout percentage for the second claim. I need to think about this a bit.

Thanks for the feedback guys!

1 Like