Partner Online Judging

Partner Judging consists of two rounds: Online Judging and Matchmaking. Each round helps partners and MassChallenge identify startups that will be able to best solve your organization's biggest challenges. In addition to being a how-to guide, this is also an overview of roles and responsibilities throughout the process. We thank you for taking the time to review and being a part of this important process!

In this article

  1. Overview
  2. Roles
  3. Adding and Removing Judges
  4. Judge Instructions
  5. View Judge Comments
  6. Review Status
  7. Evaluate

Overview

Partner Online Judging allows many judges from a single organization to provide feedback and scores for the startups to which they are assigned to evaluate. The Champion Admin(s) for each partner can review all of the feedback and aggregate scores and has the final call over which startups should advance to the Matchmaking round. This round takes place over the course of several days or weeks and each judge can participate as their own time allows.

Matchmaking uses a similar format to Partner Online Judging: a Champion Admin(s) assigns members of their team to review startup applications. Two key differences are that this round takes place over one ore two days, usually synchronously in person or via video conference, and that judges assign no scores to a startup. The Champion Admin collects feedback and makes a single decision of whether to extend an offer to a startup.

Roles

Partner Administrator

As a Partner Administrator, you will be able to edit anything to do with your Organization.

As a Partner Administrator you can: 

  • Edit your Partner Organization Name
  • Edit the logo
  • Edit the tagline
  • Add or remove team members
  • Grant Champion Administrator privileges to team members via the Edit Team Members page

Champion Administrator

As your organization's Champion Administrator, you will be the key point person, communicating both with your Partner Administrator, Champion Judges and MassChallenge staff.

As a  Champion Administrator you can:

  • Assign or remove team members to evaluate startups in a partner judging round 
  • Edit judge instructions
  • Set flag to indicate a startup should proceed to Matchmaking
  • Set flag to indicate a startup should be extended an offer after Matchmaking
  • View all scores and comments from Champion Judges in your organization
  • Assign applications to team members who are participating in the judging round

Champion Judge

As a Champion Judge,  you will evaluate startup applications assigned to you for a specific challenge area.

Assigning Applications for the Judging Round

Your MassChallenge staff contact will select the startup applications for your team to review based on the challenge areas your organization has submitted. Startup applications are assigned to your judging round on Accelerate. The Champion Administrator can see upcoming judging rounds as well as active rounds in the drop down menu. Champion Judges will only see active rounds.

Partner online judging Champion admin view dashboard
The Judging Round overview, as seen by a Champion Administrator.

Managing your Judging Round

The Champion Administrator can gain an overall view of the progress of a judging round from the main dashboard as well as evaluate applications themselves.

From here you can:

  • select judging rounds from the dropdown menu
  • view the start and end dates of a particular judging round
  • see which applications have been assigned to a given judging round
  • get an overview of the applications matchability and recommendation scoring
  • see your own review status (Champion Administrators can choose to evaluate or not)
  • manage judge assignments for any given application
  • view Matchmaking status (for Online Judging rounds)
  • view Offer Letter status (for Matchmaking rounds)

The Champion Administrator will only see team members who are active on Accelerate and a part of their team. If you do not see someone who was invited to judge, please contact your MassChallenge staff contact. Alternatively, the Champion Administrator may visit accelerate.masschallenge.org/mypartners and go to the Edit Team Members page to make appropriate changes. If you are not a Champion Administrator, you will get a prompt to contact a MassChallenge staff member for assistance.

Adding/Removing Judges

To add judges to an application, first, select the startup/application that you want to assign judges to.

Click the "View Progress and Manage Judges" button.

Click on the "Add judge" dropdown. Here you will see all active, participating team members who are available to be assigned.

Add judge screen
A list of active, participating judges, seen in the "add judges" dropdown.

Select a single judge, or multiple judges and click the blue "Add judge" button to add them.

Select a single, or multiple judges by clicking on their names in the dropdown menu.

To remove a judge, click the 'x' next to their name. 

To remove a Judge who has already been assigned, click the red "Remove Judge" link on the right side of the screen. 

If you delete a judge from an application after they have already left comments and ratings you will lose that data permanently. No information is stored if you delete.

remove judge and their feedback warning
Before you remove a judge, a warning will appear to remind you that removal means any information from that judge for that application will be permanently deleted.

You can return to the judging round overview at any time by clicking the gray "Return to Judging Overview" button.

Edit Judging Instructions

The Champion Administrator can edit judging instructions directly in each judging round. These instructions are visible to all judges on your team.

To do so, click the gray "Judging Instructions" button and the "Edit judging instructions" window will open.

A red arrow points to the right upper portion of the screen, indicating the grey "Judging Instructions" button where Champion Administrators can edit their judge instructions for the judging round.
The gray,"Judge Instructions" button on the judging round overview page allows you to edit your judging instructions.

There will be an example of judging instructions within the window by default, but it is up to your discretion what you would like your judges to see here. Please reach out to your MassChallenge staff contact for ideas. 

To edit judging instructions, simply remove the default text and add in wording that you would like to use.

Once completed, click the blue "Save" button and the instructions will be saved.

The Champion Administrator can edit these instructions at any time.

A pop up window will appear titled, "Edit Judging Instructions". Champion administrators can add their custom instructions for every individual judging round.
A window will display where judging round instructions can be edited at any time.

Review Status

The Review status allows users to see what step they are at during the evaluation process for Partner Online Judging. Review status does not apply to Matchmaking.

The Champion Administrator can see the review status for all judges as they work through their evaluations. Status options are:

  • Not Started - the user has not clicked the "Evaluate" button on an application yet.
  • In Progress - indicates that the evaluation has begun and the user has saved partial information on an application.
  • Complete - the user has finished evaluating an application and has filled out all required fields (Matchability, Recommendation and Comments) and clicked "Save."
Status indicates the state a review is in. Not started (red), in progress (yellow), complete (blue).

View Comments

Only the Champion Administrator can view all comments and scores from Partner Online Judging. There are no Champion Judge scores nor comments in the Matchmaking round.

Comments are purely internal notes for the benefit of the Partner and are not visible to the startups or other Champion Judges.

To view comments, click the blue, "View Comments" link to the right of the Judge's name, under the application that you are reviewing.

Champion Administrators can view comments left by judges.

Evaluate a Startup

Both the Champion Administrator as well as Champion Judges can evaluate startups. You can also view your evaluations and edit any submissions up to the close of a judging round.

To begin an evaluation, click the blue "Evaluate" button on the Startup application that you would like to evaluate.

Note: During Matchmaking rounds, Champion Judges will see a button that says View Application because they may only view, but not evaluate, applications at this stage.

The blue "evaluate" button is visible on the right side of the screen, across from the startup name and information.

The next screen following the screenshot above will be a combination of the evaluation panel, the full startup profile as well as their application.

On the right, you will be able to view the startup's full profile.

On the left, in the evaluation panel, you will see:

  • Startup Name
  • Challenges (each bullet point is a unique challenge)
  • View judge instructions button
  • Matchability choices
  • Recommendation choices
  • Send to Matchmaking toggle (Champion Administrators only)

Note: During Matchmaking rounds, Champion Judges will only see the Challenges listed and Champion Admins will see Challenges, Comments, and a toggle to extend an offer letter.

You can click the gray "Close" button at any time to exit out of the evaluation view. Click "Save" to save changes and remain on the Evaluate screen or click "Save & Close" to save changes and return to the overview screen.

Using the save review button is the only way to save information. If you navigate away from the evaluation page before you have saved, the data will be lost.

Evaluate view,matchability rating,recommendation,view startup profile, comments panel, send to matchmaking toggle
The evaluation page, displaying judge input fields, as well as all relevant startup information.

Every startup will be evaluated for "Matchability", "Recommendation" and "Comments"; all these fields are required to be completed.

Matchability - This is a measure of how well a startup could align with one of your challenge areas. A startup may not be a perfect fit, but you should give strong scores to companies with technology that you think may have some applicability to your organization.

Recommendation - Here, you can select your best recommendation as to how good a fit a startup is for your challenge areas and organization overall. 

Comments - Comments are purely for internal use for your company. The Champion Administrator will be able to view them and any judge can edit their comments at any time.

Send to Matchmaking toggle - Only the Champion Administrator can see and use this toggle to indicate a startup that they would like to flag for Matchmaking. When this is toggled on, the Champion Administrator will see a "Send to Matchmaking" indicator on the judging round main dashboard. No startups will be notified by this indicator; MassChallenge staff will use it to follow up with each Champion Admin.

Extend offer letter - Only Champion Admins will see this during the Matchmaking round. Use this toggle to track which startups your organization wants to extend an offer letter to. No startups will be notified by this indicator; MassChallenge staff will use it to follow up with each Champion Admin.

matchmaking toggle send to matchmaking toggle
Send to Matchmaking toggle is on, indicating a desire for this startup to be advanced to the Matchmaking round.

In the image below you can see the "Requested for Matchmaking" indicator, which will allow Champion Administrators to quickly see which startups they have flagged to move ahead. A similar indicator will appear in the Matchmaking round to indicate which startups have been flagged to receive an offer letter.

Thank you again for taking the time to review and be a part of Online Partner Judging. Please reach out to your MassChallenge staff contact for any questions or concerns.