Cortex's Scorecards feature one of the most powerful ways to promote accountability among teams and enforce standardization across services and resources. 

Scorecards gather and organize existing data within Cortex to provide greater insight into the performance of your service architecture. You can use Scorecards to establish best practices, track migration, or define maturity standards. For more information about how to use Scorecards, check out the Real World Examples article in our Help Desk.

Before using Scorecards, make sure you have services and resources in your catalogs — Scorecards evaluate services and resources, so if none exist, you won’t get much out of a Scorecard. You should also make sure essential integrations are set up, like Git tools.

Creating a Scorecard

Navigate to Scorecards through the tab in the navigation bar.

As with other features in Cortex, you’ll see all of your organization's Scorecards populate under All, while any Scorecards related to services and resources you own will appear under Mine.

Those with the admin or manager permission levels will see the option to Create Scorecard.

Starting with a template

When you create a new Scorecard, you have the option to use a Scorecard template or Start from scratch.

To work with a Scorecard template, select any of the Scorecard templates built into Cortex, and click Continue.

From there, you’ll be taken to an overview page where you can view the Integrations used by the Scorecard, as well as the template’s Levels and rules.

On the next page, you have the opportunity to edit the Scorecard's Name and Description. At this stage, you'll also determine which services or resources the Scorecard applies to.

Before saving your Scorecard, you can also make adjustments to the template's predefined levels.

The draft toggle allows you to keep your Scorecard private until it’s ready to be viewed more widely. While in draft status, your Scorecard will not appear in reports or send notifications, and it’ll only be visible to admins and managers, so it won’t be noisy for developers. 

If you’re testing a Scorecard, or aren’t ready for others to see it, keep the draft toggle engaged.

Starting from scratch

To Start from scratch, select that option from Getting started, and click Continue.

Whether you create a Scorecard from scratch or build it from a template, you'll be asked to enter a name, tag, and description for your Scorecard. At this stage, you’ll also decide whether your Scorecard will evaluate services or resources, and apply filters to those services or resources.

Keep in mind that the Scorecard's name and description will be visible across Cortex, so make sure it’s obvious what services or resources are being scored on. For example, you may name your Scorecard “Security” — it’s pretty clear what that Scorecard is designed to measure. Beyond that, you may link to a confluence wiki page in the description, as well as a brief summary of how you intend to use the Scorecard. The description features full markdown support, so you can enter anything that is relevant in that field.

Adding rules

After entering metadata about your Scorecard, you’ll Define evaluation rules, the criteria your services or resources will be scored against.

You have two ways to set up your rules. We recommend using Level progression, which allows you to create establish progressive levels as you’re adding rules. (You can read more about Ladders in the next help desk article.) You can also add Point-based rules, where each rule has a weighted value.

Regardless of which option you choose, you’ll have two ways to write your rules. You can use the Form editor, choose an integration, and follow the flow in the UI.

Or, if you’re familiar with Cortex Query Language (CQL), you can toggle the CQL Editor and write your own expression. This option will make more sense when writing complex rules that rely on custom data.

Most of the time, it’ll make sense to work with an integration, especially in the beginning. This way, you don’t have to spend time upfront reading through the CQL docs. This is one of the reasons we recommend setting up integrations before creating a Scorecard — it's easy and intuitive to build rules based on integrations.

Once you've defined your rule, you also have the ability to add a human-readable title and a description. If you do not add a title, your rule will display as a CQL expression — for this reason, we strongly recommend adding a title.

At this stage, you also have the ability to add a failure message, which provides an additional opportunity to point developers in the right direction. The failure message will display any time a service or resource is not passing a rule, so developers have a clear sense of how to remedy the issue.

While both the description and failure message are optional, including short messages will allow you to make the most of this powerful feature.

Level progression

If you choose to Define evaluation rules using Level progression, you have the ability to tier your rules and set priorities, making it clear to developers which goals should be accomplished first.

You can always create a ladder later, so if you’re not sure how you’d like to prioritize your rules, you can skip this step.

For more information on ladders, you can read this article on Scorecard ladders and levels. The process is exactly the same at this stage as it is when you’re editing a Scorecard, so you can follow the same steps.

Filtering

One of the key steps in building a Scorecard is filtering, deciding which services or resources the Scorecard will apply to. By default, a Scorecard will apply to all services or resources in the catalog, but you have the ability to narrow your selection down to more specific subsets, like front-end libraries or tier-1 services.

If you decide to filter services or resources for your Scorecard, you’ll first decide whether to apply Basic or Advanced filtering. With Basic filtering, you’ll use the Cortex UI to determine which groups the Scorecards should apply to, and which groups should be excluded. 

For a resource-based Scorecard, you also have the option to include or exclude by resource type, such as s3 or RDS. Note that the toggle above Applies to types determines inclusion or exclusion.

Advanced filtering allows users to write CQL expressions that dictate what is included or excluded from a scorecard. You can write your query within the field provided or use the CQL Builder to construct your filter.

Note that advanced filtering only works with data that exists within Cortex (such as service groups, owners, and other custom data), but does not extend to third party integrations. 

Editing a Scorecard

To edit a Scorecard, click Edit within any Scorecard's home page.

Note: Just like with creating a Scorecard, only admins and managers will have the ability to edit a Scorecard.

The Edit Scorecard page is identical to the Create Scorecard page. You can modify your Scorecard’s name and description, edit levels, add or delete rules, and toggle the draft status. You can also adjust the filter criteria and change the services or resources being evaluated by the Scorecard to fine-tune your insights over time.

Any time you edit and save your Scorecard, Cortex will automatically re-evaluate the relevant services and resources to reflect your changes.