Missing simple measures in probe dictionary

[godelstheory]

Searching for the following simple measures in the probe dictionary finds no probes:

• selectProfile
• delayedStartupStarted
• delayedStartupFinished
• sessionRestored
• blankWindowShown

However, these are all available in about.telemetry (Simple Measures) and are observed in raw pings for current Firefox release data

from moztelemetry.dataset import Dataset
from pyspark.sql import functions as F, types as T

records = (Dataset
     .from_source("telemetry")
     .select(client_id='clientId')
     # .select(**{attr: 'payload.simpleMeasurements.{}'.format(attr) for attr in attributes})
     . select('payload.simpleMeasurements')
     .where(docType='main', 
            submissionDate='20181201',
            appUpdateChannel='release',
            appVersion=lambda x: x.startswith('64')
           )
     .records(sc, sample=0.01))

records.first()

yields:

{'client_id': u'0ba3abfd-3812-4dca-861d-36e394a0d77a', 'payload.simpleMeasurements': {u'AMI_startup_begin': 433, u'AMI_startup_end': 473, u'XPI_bootstrap_addons_begin': 449, u'XPI_bootstrap_addons_end': 467, u'XPI_finalUIStartup': 495, u'XPI_startup_begin': 444, u'XPI_startup_end': 467, u'activeTicks': 46, u'addonManager': {u'XPIDB_parseDB_MS': 13}, u'afterProfileLocked': 317, u'blankWindowShown': 418, u'createTopLevelWindow': 364, u'debuggerAttached': 0, u'delayedStartupFinished': 830, u'delayedStartupStarted': 794, u'firstPaint': 701, u'main': 307, u'maximalNumberOfConcurrentThreads': 80, u'profileBeforeChange': 361540, u'quitApplication': 361383, u'selectProfile': 317, u'sessionRestoreInit': 494, u'sessionRestoreInitialized': 497, u'sessionRestored': 1232, u'shutdownDuration': 558, u'start': 25, u'startupCrashDetectionBegin': 422, u'startupCrashDetectionEnd': 31688, u'startupInterrupted': 0, u'startupSessionRestoreReadBytes': 116863447, u'startupSessionRestoreWriteBytes': 192757, u'startupWindowVisibleReadBytes': 110185984, u'startupWindowVisibleWriteBytes': 65896, u'totalTime': 362}, 'version': u'64.0'}

[chutten]

sessionRestored does appear to be there, but you're right that some are missing.

The rest need to be added to other_fields.json

[hardik25]

I would like to take this up. May I? If yes, could you please provide some leads?

[chutten]

@hardik25 's fix in #614 took care of this issue.