Important Dates to Remember

• Schedule Announcement: Wednesday, November 20
• Event Dates: Tuesday, March 3 – Friday, March 6
• Review Period Closes: Thursday, October 31
• Review Call: Week of November 4

Scoring Guidelines 

Grade each session on a 5 to 1 grading scale:

• 5 (strong accept)
• 4 (accept)
• 3 (average)
• 2 (reject)
• 1 (strong reject)

Review Process Best Practices

1. Process Integrity: It is very important to protect the integrity of the review process, and avoid the impression of a biased review. Please review and adhere to our Code of Conduct. 
2. Public & Author Interaction: To ensure an unbiased review process, program committee members should not discuss submissions with authors and/or the overall public (i.e., please no tweeting) while sessions are being reviewed, particularly before acceptance/rejection notices have been sent.
3. Conflict of Interest: Reviewers are asked to wear their “Hyperledger” hats first when scoring submissions, and rate all submissions fairly. If a submission was written by a colleague, or someone that you are seen to be associated with or in competition with, please do your best to be as unbiased as possible.
4. Review Metrics: As listed above, the ranking system is divided into 5 options: 5 (Strong Accept), 4 (Accept), 3 (Average), 2 (Reject), 1 (Strong Reject). It is important that you highlight your level of confidence in your recommendation and the reasons why you gave the score you did. When reviewing proposals, keep in mind the following criteria:
   1. Relevance
   2. Originality
   3. Soundness 
   4. Quality of Presentation
   5. Importance

Based on the feedback from previous editions, the PC should bear in mind the following guidance:

• Ensure that the content presented does not contain pitches, inappropriate content and will be useful to the audience
• Ensure that the speakers are coming from diverse backgrounds in all possible terms
  • Gender
  • Geography
  • Companies
  • Race
  • Etc
• While the first phase of the review process will be blind, the PC will then revisit accepted talks to ensure the diversity of speakers and companies. 
• Avoid long talks – preferably 30 min or less, unless they are engaging the audience (eg. a live demo or a Q&A session.
• Encourage single presenter talks and ensure that panel submissions have no more than 3 panelists and a moderator. Prioritize the quality over quantity.
• Accommodate as many speakers as possible without reducing the value of the event to the audience
• Encourage interactive content – short talks with long Q&A, demos, live coding sessions
• Presentations may be accepted on a conditional basis. If a presentation seems interesting but would benefit from a change of format or improvement to the content, the PC can decide to appoint one of its members to work with the submitter to improve the content and then fully accept it.

Review Comments: Keep in mind that submitting authors may be a VP at a large company or a university student. Ensure your feedback is constructive, in particular for rejected proposals as we do receive requests for feedback and it is always helpful to pass along helpful comments for future proposals. Good examples of review elements include:

• Highlighting the positive aspects of a proposal
• Providing constructive feedback, “It would have been helpful if…” and include facts when applicable
• Refrain from any personal comments.

If you require any assistance reviewing proposals or have questions about the review process or any of the best practices we have suggested, please contact Shannon Jessee sjessee@linuxfoundation.org for assistance.