Normalizing Dumps
nosoop edited this page 2 years ago

Why the need for normalization?

The dump process runs on my production server, so there's some effort done to ensure diffs are consistent regardless of how the server is configured.

This page assumes that you're processing the dumps on a device with common (for Linux) shell scripting utilities.

Built-In SourceMod Dumps

Since it involves engine functionality, this one's fairly consistent — no plugins can really mess around with the offsets of networked properties and entity data. Only inconsistent thing would be the SourceMod comments (which contains a datestamp), which can be stripped out of the datamaps / netprops / classes files with the following function:

# strip out leading comments and empty lines
grep -vP '^(//|$)' "$@"

Recreating SourceMod's Class Dumps

A more recent version of the plugin removed the use of sm_dump_classes. The commands used to dump it and datamaps require one of each entity to be spawned, which allows us to reduce the possibility of hitting the 2048 entity limit.

The entity names in the datamaps dump has the exact same format that the classes dump has, so we can just filter through the datamaps file to remake it.

grep -P '^\w+ - \w+$' "${datamaps_file}" > "${classes_file}"

ConCommand Dumps

The plugin uses a custom dump process that does the following:

  • Maintains default values of ConVars (whereas cvarlist uses current values).
  • Preserves iteration order (it looks like it goes from most recently created to least recent).

The latter is especially relevant, as engine and server ConVars and ConCommands are created before external plugin ones. Unless you have another Valve Server Plugin (such as Botrix), MetaMod:Source is likely to load first, creating the meta command and the meta_version ConVar.

As a result, we can normalize both dumps with the following:

# print everything after either concmd "meta" or convar "metamod_version"
    
# https://stackoverflow.com/a/17914105 (see B)
awk 'f;/^"meta(mod_version)?"/{f=1}' "$1" | sort

Stripping Offsets

Datamap / netprop dumps note the offset of a member. While useful to see if any non-visible members are inserted / removed, it does clutter up the result of diff.

You can strip the offset value like so:

sed "${input}" -e 's: (offset [0-9]\+)::'