MozillaURLProvider.py throwing CERTIFICATE_VERIFY_FAILED errors

[jps3]

Describe the problem Since sometime after Merge Request #348 I and some others have reported seeing [SSL: CERTIFICATE_VERIFY_FAILED] errors from our AutoPkg runs.

Processor: MozillaURLProvider: Error: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)

Preferences contents

 {
     "API_PASSWORD" = "...snip...";
     "API_USERNAME" = "...snip...";
     "CACHE_DIR" = "/Users/madmin/Library/AutoPkg/Cache";
     "FAIL_RECIPES_WITHOUT_TRUST_INFO" = 1;
     "GIT_PATH" = "/Applications/Xcode.app/Contents/Developer/usr/bin/git";
     "JSS_REPOS" =     (
        ...snip... 
     );
     "JSS_URL" = "...snip...";
     "JSS_VERIFY_SSL" = 1;
     "MUNKI_REPO" = "/Volumes/RAID/munki/repo";
     "RECIPE_OVERRIDE_DIRS" = "/Users/madmin/Library/AutoPkg/RecipeOverrides";
     "RECIPE_REPOS" =     {
        ...snip...
      };
     "RECIPE_REPO_DIR" = "/Users/madmin/Library/AutoPkg/RecipeRepos";
     "RECIPE_SEARCH_DIRS" =     (
        ...snip...
     );
     "VIRUSTOTAL_SLEEP_SECONDS" = 15;
 }

AutoPkg output See gist

Expected behavior No TLS/SSL certificate error.

Version (please complete the following information):

• OS version: 10.14.6 (18G6020)
• AutoPkg Version: 2.1

[jps3]

I have a proposed fix which will be submitted as pull request shortly.

[jps3]

Sorry for edit of title. I went to double check accuracy of submission and could not figure out where I'd gotten the "348" from and I did not want to confuse the issue.

[jps3]

Test of pull request.

$ autopkg run -vv --ignore-parent-trust-verification-errors local.munki.Firefox
Processing local.munki.Firefox...
MozillaURLProvider
{'Input': {'locale': 'en-US', 'product_name': 'firefox', 'release': 'latest'}}
MozillaURLProvider: No value supplied for platform, setting default value of: osx
MozillaURLProvider: No value supplied for base_url, setting default value of: https://download.mozilla.org/?product={product_release}-ssl&os={platform}&lang={locale}
MozillaURLProvider: No value supplied for versions_base_url, setting default value of: https://product-details.mozilla.org/1.0/{product}_versions.json
MozillaURLProvider: Found URL https://download.mozilla.org/?product=firefox-latest-ssl&os=osx&lang=en-US
{'Output': {'moz_original_version': '79.0',
            'moz_version': '79.0',
            'url': 'https://download.mozilla.org/?product=firefox-latest-ssl&os=osx&lang=en-US'}}
URLDownloader
{'Input': {'filename': 'Firefox.dmg',
           'url': 'https://download.mozilla.org/?product=firefox-latest-ssl&os=osx&lang=en-US'}}
URLDownloader: No value supplied for prefetch_filename, setting default value of: False
URLDownloader: No value supplied for CHECK_FILESIZE_ONLY, setting default value of: False
URLDownloader: Item at URL is unchanged.
URLDownloader: Using existing /Users/madmin/Library/AutoPkg/Cache/local.munki.Firefox/downloads/Firefox.dmg
{'Output': {'pathname': '/Users/madmin/Library/AutoPkg/Cache/local.munki.Firefox/downloads/Firefox.dmg'}}
EndOfCheckPhase
{'Input': {}}
{'Output': {}}
CodeSignatureVerifier
{'Input': {'DISABLE_CODE_SIGNATURE_VERIFICATION': False,
           'input_path': '/Users/madmin/Library/AutoPkg/Cache/local.munki.Firefox/downloads/Firefox.dmg/Firefox*.app',
           'requirement': 'anchor apple generic and certificate '
                          'leaf[field.1.2.840.113635.100.6.1.9] /* exists */ '
                          'or anchor apple generic and certificate '
                          '1[field.1.2.840.113635.100.6.2.6] /* exists */ and '
                          'certificate leaf[field.1.2.840.113635.100.6.1.13] '
                          '/* exists */ and certificate leaf[subject.OU] = '
                          '"43AQ936H96"'}}
CodeSignatureVerifier: Mounted disk image /Users/madmin/Library/AutoPkg/Cache/local.munki.Firefox/downloads/Firefox.dmg
CodeSignatureVerifier: Using path '/private/tmp/dmg.lzrpRv/Firefox.app' matched from globbed '/private/tmp/dmg.lzrpRv/Firefox*.app'.
CodeSignatureVerifier: Verifying code signature...
CodeSignatureVerifier: Deep verification enabled...
CodeSignatureVerifier: Strict verification not defined. Using codesign defaults...
CodeSignatureVerifier: /private/tmp/dmg.lzrpRv/Firefox.app: valid on disk
CodeSignatureVerifier: /private/tmp/dmg.lzrpRv/Firefox.app: satisfies its Designated Requirement
CodeSignatureVerifier: /private/tmp/dmg.lzrpRv/Firefox.app: explicit requirement satisfied
CodeSignatureVerifier: Signature is valid
{'Output': {}}
MunkiImporter
{'Input': {'MUNKI_REPO': '/Volumes/RAID/munki/repo',
           'pkg_path': '/Users/madmin/Library/AutoPkg/Cache/local.munki.Firefox/downloads/Firefox.dmg',
           'pkginfo': {'catalogs': ['testing'],
                       'category': 'Web Browsers',
                       'description': 'Mozilla Firefox is a free and open '
                                      'source web browser.',
                       'developer': 'Mozilla',
                       'display_name': 'Mozilla Firefox',
                       'name': 'Firefox',
                       'unattended_install': True},
           'repo_subdirectory': 'apps/firefox'}}
MunkiImporter: Item Firefox.dmg already exists in the munki repo as pkgs/apps/firefox/Firefox-79.0.dmg.
{'Output': {'pkg_repo_path': '/Volumes/RAID/munki/repo/pkgs/apps/firefox/Firefox-79.0.dmg'}}
Receipt written to /Users/madmin/Library/AutoPkg/Cache/local.munki.Firefox/receipts/local.munki-receipt-20200730-115555.plist

Nothing downloaded, packaged or imported.

[smashism]

Hello, since getting the update to MozillaURLProvider.py I'm getting syntax errors when our Firefox policy runs.

Traceback (most recent call last):
  File "/usr/local/bin/autopkg", line 2810, in <module>
    sys.exit(main(sys.argv))
  File "/usr/local/bin/autopkg", line 2804, in main
    exit(subcommands[verb]["function"](argv))
  File "/usr/local/bin/autopkg", line 2323, in run_recipes
    autopackager.verify(recipe)
  File "/Library/AutoPkg/autopkglib/__init__.py", line 606, in verify
    step["Processor"], recipe=recipe, env=self.env
  File "/Library/AutoPkg/autopkglib/__init__.py", line 822, in get_processor
    _tmp = imp.load_source(processor_name, processor_filename)
  File "/Users/Shared/AutoPkg/RecipeRepos/com.github.autopkg.recipes/Mozilla/MozillaURLProvider.py", line 27
    __all__: List[str] = ["MozillaURLProvider"]
           ^
SyntaxError: invalid syntax

Is this a python2 vs python3 thing? We're still running autopkg 1.4.1.

[nmcspadden]

The core issue was fixed by #351, so I'm closing this.

@smashism unfortunately, as a result, that means the recipes are no longer compatible with 1.x.

[nmcspadden]

I have pushed commit 10541f07e781358fc489c1ed615de9ce85ee95a5 to update the recipes to require 2.0 as a minimum version.