Abstract

The following are sets of rules that, in my opinion, need to be implemented in the Security Department of Callisto to ensure full functionality. This proposal should come into effect at 15th July, 2019.

Motivation

Callisto is intended to be a decentralized auditing platform. The process of hiring is still a centralized aspect of the project which needs to be updated. The main purpose of this changes is to make Callisto Auditing Department open for contributions and more flexible.

Specification

The main goal of this proposal is to allow everyone to participate in Security auditing of contracts and getting paid as third party auditors.

I propose to deprecating the procedure of hiring through test-tasks. Instead, we should allow everyone to participate and show their skills in real contract auditing and then become an approved auditor.

Auditors and Auditing Manager

There will be three types of participants in Callisto Security Department:

Auditing Manager
Approved auditor.
Third party auditor. ("freelance" auditor)

Auditing Manager must:

follow the rules of the Callisto Security Department and ensure that auditors follow the rules.
assign auditors to tasks if auditors have requested an assignment to perform a an audit.
compare audit reports submitted by assigned auditors and finish an audit request if all the assigned auditors have provided their reports.
fork auditors report-gists and publish them after the completion of each audit.
publish the audit summary after the completion of an audit according to the disclosure policy.
calculate the security auditor's scores at 15th of each month.

Auditing Manager may:

reject auditors request for assignment if (1) this audit is not approved, (2) there are enough auditors assigned for the task, (3) the auditor is currently assigned to another task and this task is not completed.
close/withdraw dulicate tasks.
notify contract developers of critical findings in their contracts.
notify the contract developer of the need for actions at the end of the contract audit, i.e. implementation of certain changes, bug fixing, committing a bug bounty.

Approved or third-party auditor may:

request an assignment for an audit by commenting a corresponding issue with approximate time required for the audit performance.

Approved or third-party auditor must:

create a secrect gist and send it to the auditing manager by email after the assignment to a task.
perform a security audit of a project and describe each finding in the report-gist.
if an auditor has completed the verification of the contract, he should write about this in the comments of the corresponding audit.

Salaries

Auditors receive salaries based on the auditing score. Salary calculation is described here.

Approved auditors receive a full amount of calculated salary. Third party auditors receive 75% of the calculated salary.

If the contract audit cannot be completed, since there are not enough auditors to work on this contract, then 70% of the salary is paid to the auditors who worked on this contract, as if they had found all the errors in this contract. If the audit is subsequently completed, the difference between what should be paid for this audit is either paid to the auditors at the time of the next salary, or withheld from their next salary if they missed any serious errors.

Salary Withholding

In some cases, it is necessary to impose fines on auditors. In Callisto, this is accomplished by withholding a portion of the next salary of the auditor.

The amount withheld cannot exceed 50% of the monthly salary of the auditor and cannot reduce the salary of the auditor below the established minimum ( $500 ).

Salary will be withheld if:

all auditors missed an error during the audit and did not describe it. In this case, the difference between the previously paid salary and the value that should have been paid, taking into account the error found afterwards, will be withheld.
a hack occurs on an audited contract. In this case, the full amount of salary previously paid for the audit of this contract will be withheld.
an auditor violated the rules of Callisto Security Department which includes but is not limited to (1) formatting of the audit report-gist, (2) commenting the completion of an audit, (3) failing to complete the audit at time. 10% of the audit reward may be withheld from the next salary. It is determined by the security auditing manager to impose these fines or not in each individual case.

Becoming an approved auditor

A third party auditor must fulfill two criterias to become an approved auditor:

Perform audits of at least 3 contracts.
Perform audits of at least 1200 lines of code.

Once the described criterias are fulfilled, the third party auditor may apply for the position of an approved auditor.

IMPORTANT: If a third party auditor has fulfilled the approvement criterias and become an approved auditor before his first salary payment then his first three contracts are evaluated as audited by approved auditor. This auditor will receive a full amount of salary at the salary payment day.

Assigning auditors

General rules:

Each smart-contract must be reviewed by at least two approved auditors.
Each smart-contract must be reviewed by at least three auditors in total.
High priority smart-contracts must be reviewed by at least three approved auditors.

An auditing manager must assign auditors to ensure compliance with the described general rules.

Assigning an approved auditor

Approved auditor may be assigned to the task if:

he requested an assignment to the task.
there are less than three auditor assigned to the task currently.
the task is "approved" for assignment and there are no tasks with higher priority in queue.

Assigning a third party auditor

Third party auditor may be assigned to the task if:

he requested an assignment to the task.
the task is "approved" for assignment.
the time that the third party auditor has commented on does not exceed the estimated time to complete this audit by the last of the approved auditors, by more than three days.
there are less than four auditor assigned to the task currently (normal or low priority audits).
there are less than five auditor assigned to the task currently (high priority audits)

EthereumCommonwealth / Roadmap

Auditing Department update: hiring procedure and 3d party involvement. #59