How to scrub sensitive data

Set up scrubbers using RE2 to avoid logging sensitive data.

Scrubbers help to redact sensitive data exposed in commands, command output (stdout, stderr), and file diffs.

To access the interface where you can define scrubbers:

  • Click your project’s title to open the dropdown menu.
  • Click Project & app settings, then choose Agent on the left panel:

Next, to define a scrubber:

  • Click Add scrubber to open this menu:

  • Name the scrubber, then select the data type it will apply to (“Command”, “File diff”, or “Output”).
  • Consider unchecking the option Activate scrubber for new scrubbers while you thoroughly test them.
  • Cmd scrubbers use RE2. Define the scrubber using this syntax.
  • Wrap the data you want scrubbed in one or more capture groups.
  • Test the scrubber by using the `Test string` section.
  • The input field (on the left) is editable.
  • The output field (on the right) shows the contents of the input field after passing through the scrubber defined above.
  • When you are satisfied with your scrubber, click Save.

Example

This screenshot shows a scrubber which redacts the passwords in unsafe Mysql commands:

Scrubber suggestions

We hope this table of working RE2 scrubbers provides some helpful starting points or inspiration.

Potential Data Leak RE2 regular expression Scrubber description
American Express credit card numbers (\b\d{13,16}\b) Captures numbers which are 13-16 characters in length, with no spaces, surrounded by whitespace
Access tokens --token[=\s]?(\S*) Captures the first work after “–token " or “–token;”.
Passwords --pass[=\s]?(\S*) Captures the first work after “–pass " or “–pass;”.
Private SSH keys ([A-Za-z0-9/+]{64,128}) Captures strings comprised of characters valid in SSH keys, between 64 and 128 bits in length.

Limiting line length
Cmd scrubbers read line by line. The Change line length option allows you to set an upper bound on how much data from each line will undergo scrubbing at once. The ‘line length’ setting defaults to 4,096 bytes, and its upper bound is 65,536 bytes; any part of a line that extends beyond it will still be scrubbed, but as part of a new line. This setting does not apply to Command scrubbers, only to output and file diff scrubbers.

Additional resources

RE2 documentation