search

machawk1 / wail

:whale2: Web Archiving Integration Layer: One-Click User Instigated Preservation
https://matkelly.com/wail
MIT License
349 stars 35 forks source link

Heritrix jobs are neither started nor info available on OS X 10.9 #229

Closed machawk1 closed 5 years ago

machawk1 commented 8 years ago

screen shot 2015-10-23 at 12 04 53 pm

machawk1 commented 8 years ago

The crawler bean XML file is created but looks to never be built nor fired off.

machawk1 commented 8 years ago

The Heritrix API looks to be not responding nor accepting the default credentials:

INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): localhost
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEIbtY3fO/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"POST /engine/job/1445627386 HTTP/1.1" 401 312
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEIbtY3fO/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"POST /engine/job/1445627386 HTTP/1.1" 404 161
<Response [404]>
{'date': 'Fri, 23 Oct 2015 19:09:49 GMT', 'content-length': '161', 'content-type': 'text/html; charset=ISO-8859-1', 'accept-ranges': 'bytes', 'server': 'Noelios-Restlet-Engine/1.1.10'}
<h1>Page not found</h1>
The page you are looking for does not exist.  You may be able to recover by going <a href='javascript:history.back();void(0);'>back</a>.

Launching heririx job
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): localhost
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEIbtY3fO/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"POST /engine/job/1445627386 HTTP/1.1" 401 312
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEIbtY3fO/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"POST /engine/job/1445627386 HTTP/1.1" 404 161
<Response [404]>
{'date': 'Fri, 23 Oct 2015 19:09:49 GMT', 'content-length': '161', 'content-type': 'text/html; charset=ISO-8859-1', 'accept-ranges': 'bytes', 'server': 'Noelios-Restlet-Engine/1.1.10'}
<h1>Page not found</h1>
The page you are looking for does not exist.  You may be able to recover by going <a href='javascript:history.back();void(0);'>back</a>
machawk1 commented 8 years ago

Despite Heritrix being able to see the file-based structure of the jobs, it does not recognize any jobs to build. screen shot 2015-10-23 at 12 12 27 pm

screen shot 2015-10-23 at 12 13 16 pm

machawk1 commented 8 years ago

This occurs even with the system Firewall off.

machawk1 commented 8 years ago

KILLing then FIXing Heritrix via the WAIL UI allows crawl jobs to be fired off as expected. Hmm.

Writing path-index.txt file
Done writing path-index.txt file
CDX files generated for each WARC, merging...
Done merging CDX files, removing old source CDX files.
Sorting CDX entries
Removing unsorted temp file
Done creating sorted CDX file!
Trying to access Heritrix service at https://127.0.0.1:8443
Pseudo-Success in accessing https://127.0.0.1:8443
Trying to access Wayback service at http://localhost:8080/wayback/
Wayback is a go! 
Trying to access Heritrix service at https://127.0.0.1:8443
Pseudo-Success in accessing https://127.0.0.1:8443
Trying to access Wayback service at http://localhost:8080/wayback/
Wayback is a go! 
Using CATALINA_BASE:   /Applications/WAIL.app/bundledApps/tomcat
Using CATALINA_HOME:   /Applications/WAIL.app/bundledApps/tomcat
Using CATALINA_TMPDIR: /Applications/WAIL.app/bundledApps/tomcat/temp
Using JRE_HOME:        /Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home
Using CLASSPATH:       /Applications/WAIL.app/bundledApps/tomcat/bin/bootstrap.jar:/Applications/WAIL.app/bundledApps/tomcat/bin/tomcat-juli.jar
Building heririx job
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): localhost
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEI7r41wV/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"POST /engine/job/1445628035 HTTP/1.1" 401 312
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEI7r41wV/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
Writing path-index.txt file
Done writing path-index.txt file
CDX files generated for each WARC, merging...
Done merging CDX files, removing old source CDX files.
Sorting CDX entries
Removing unsorted temp file
Done creating sorted CDX file!
DEBUG:requests.packages.urllib3.connectionpool:"POST /engine/job/1445628035 HTTP/1.1" 303 0
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEI7r41wV/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"GET /engine/job/1445628035 HTTP/1.1" 401 312
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEI7r41wV/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"GET /engine/job/1445628035 HTTP/1.1" 200 None
<Response [200]>
{'transfer-encoding': 'chunked', 'accept-ranges': 'bytes', 'vary': 'Accept-Charset, Accept-Encoding, Accept-Language, Accept', 'server': 'Noelios-Restlet-Engine/1.1.10', 'date': 'Fri, 23 Oct 2015 19:20:41 GMT', 'content-type': 'application/xml; charset=UTF-8'}
<?xml version="1.0" standalone='yes'?>

<job>
  <shortName>1445628035</shortName>
  <crawlControllerState>NASCENT</crawlControllerState>
  <statusDescription>Ready</statusDescription>
  <availableActions>
    <value>launch</value>
    <value>teardown</value>
  </availableActions>
  <launchCount>0</launchCount>
  <lastLaunch/>
  <isProfile>false</isProfile>
  <primaryConfig>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/crawler-beans.cxml</primaryConfig>
  <primaryConfigUrl>https://localhost:8443/engine/job/1445628035/jobdir/crawler-beans.cxml</primaryConfigUrl>
  <url>https://localhost:8443/engine/job/1445628035/job/1445628035</url>
  <jobLogTail>
    <value>2015-10-23T19:20:41.415Z INFO Job instantiated</value>
  </jobLogTail>
  <uriTotalsReport>
    <downloadedUriCount>0</downloadedUriCount>
    <queuedUriCount>0</queuedUriCount>
    <totalUriCount>0</totalUriCount>
    <futureUriCount>0</futureUriCount>
  </uriTotalsReport>
  <sizeTotalsReport>
    <dupByHash>0</dupByHash>
    <dupByHashCount>0</dupByHashCount>
    <notModified>0</notModified>
    <notModifiedCount>0</notModifiedCount>
    <novel>0</novel>
    <novelCount>0</novelCount>
    <total>0</total>
    <totalCount>0</totalCount>
  </sizeTotalsReport>
  <rateReport>
    <currentDocsPerSecond>0.0</currentDocsPerSecond>
    <averageDocsPerSecond>NaN</averageDocsPerSecond>
    <currentKiBPerSec>0</currentKiBPerSec>
    <averageKiBPerSec>0</averageKiBPerSec>
  </rateReport>
  <loadReport>
    <busyThreads>0</busyThreads>
    <totalThreads>0</totalThreads>
    <congestionRatio>0.0</congestionRatio>
    <averageQueueDepth>0</averageQueueDepth>
    <deepestQueueDepth>-1</deepestQueueDepth>
  </loadReport>
  <elapsedReport>
    <elapsedMilliseconds>0</elapsedMilliseconds>
    <elapsedPretty>0ms</elapsedPretty>
  </elapsedReport>
  <threadReport/>
  <frontierReport/>
  <crawlLogTail></crawlLogTail>
  <configFiles>
    <value>
      <key>loggerModule.crawlLogPath</key>
      <name>crawl.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/crawl.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/crawl.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>org.archive.modules.deciderules.surt.SurtPrefixedDecideRule#1259f252.surtsDumpFile</key>
      <name>surtsDumpFile</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/negative-surts.dump</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/negative-surts.dump</url>
      <editable>false</editable>
    </value>
    <value>
      <key>actionDirectory.actionDir</key>
      <name>ActionDirectory source directory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/action</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/action</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.path</key>
      <name>logs subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.nonfatalErrorsLogPath</key>
      <name>nonfatal-errors.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/nonfatal-errors.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/nonfatal-errors.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>statisticsTracker.reportsDir</key>
      <name>reports subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/reports</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/reports</url>
      <editable>false</editable>
    </value>
    <value>
      <key>warcWriter.storePaths[0]</key>
      <name>warcWriter.storePaths[0]</name>
      <path>/Applications/WAIL.app/archives</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/archives</url>
      <editable>false</editable>
    </value>
    <value>
      <key>checkpointService.checkpointsDir</key>
      <name>checkpoints subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/checkpoints</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/checkpoints</url>
      <editable>false</editable>
    </value>
    <value>
      <key>warcWriter.defaultStorePaths[0]</key>
      <name>warcs default store path</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/warcs</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/warcs</url>
      <editable>false</editable>
    </value>
    <value>
      <key>actionDirectory.doneDir</key>
      <name>ActionDirectory done directory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/actions-done</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/actions-done</url>
      <editable>false</editable>
    </value>
    <value>
      <key>crawlController.scratchDir</key>
      <name>scratch subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/scratch</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/scratch</url>
      <editable>false</editable>
    </value>
    <value>
      <key>warcWriter.directory</key>
      <name>writer base path</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.uriErrorsLogPath</key>
      <name>uri-errors.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/uri-errors.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/uri-errors.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>bdb.dir</key>
      <name>bdbmodule subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/state</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/state</url>
      <editable>false</editable>
    </value>
    <value>
      <key>org.archive.modules.deciderules.surt.SurtPrefixedDecideRule#51cdf94.surtsDumpFile</key>
      <name>surtsDumpFile</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/surts.dump</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/surts.dump</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.progressLogPath</key>
      <name>progress-statistics.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/progress-statistics.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/progress-statistics.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.alertsLogPath</key>
      <name>alerts.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/alerts.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/alerts.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.runtimeErrorsLogPath</key>
      <name>runtime-errors.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/runtime-errors.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/${launchId}/logs/runtime-errors.log</url>
      <editable>false</editable>
    </value>
  </configFiles>
  <isLaunchInfoPartial>false</isLaunchInfoPartial>
  <isRunning>false</isRunning>
  <isLaunchable>true</isLaunchable>
  <hasApplicationContext>true</hasApplicationContext>
  <alertCount>0</alertCount>
  <checkpointFiles></checkpointFiles>
  <alertLogFilePath>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/./jobs/1445628035/${launchId}/logs/alerts.log</alertLogFilePath>
  <reports>
    <value>
      <className>CrawlSummaryReport</className>
      <shortName>CrawlSummary</shortName>
    </value>
    <value>
      <className>SeedsReport</className>
      <shortName>Seeds</shortName>
    </value>
    <value>
      <className>HostsReport</className>
      <shortName>Hosts</shortName>
    </value>
    <value>
      <className>SourceTagsReport</className>
      <shortName>SourceTags</shortName>
    </value>
    <value>
      <className>MimetypesReport</className>
      <shortName>Mimetypes</shortName>
    </value>
    <value>
      <className>ResponseCodeReport</className>
      <shortName>ResponseCode</shortName>
    </value>
    <value>
      <className>ProcessorsReport</className>
      <shortName>Processors</shortName>
    </value>
    <value>
      <className>FrontierSummaryReport</className>
      <shortName>FrontierSummary</shortName>
    </value>
    <value>
      <className>ToeThreadsReport</className>
      <shortName>ToeThreads</shortName>
    </value>
  </reports>
  <heapReport>
    <usedBytes>28577120</usedBytes>
    <totalBytes>63438848</totalBytes>
    <maxBytes>238551040</maxBytes>
  </heapReport>
</job>

Launching heririx job
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): localhost
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEI7r41wV/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"POST /engine/job/1445628035 HTTP/1.1" 401 312
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEI7r41wV/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"POST /engine/job/1445628035 HTTP/1.1" 303 0
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEI7r41wV/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"GET /engine/job/1445628035 HTTP/1.1" 401 312
/var/folders/kz/nxn4kz_s6p558hkmvdx3nqs00000gn/T/_MEI7r41wV/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
DEBUG:requests.packages.urllib3.connectionpool:"GET /engine/job/1445628035 HTTP/1.1" 200 None
<Response [200]>
{'transfer-encoding': 'chunked', 'accept-ranges': 'bytes', 'vary': 'Accept-Charset, Accept-Encoding, Accept-Language, Accept', 'server': 'Noelios-Restlet-Engine/1.1.10', 'date': 'Fri, 23 Oct 2015 19:20:43 GMT', 'content-type': 'application/xml; charset=UTF-8'}
<?xml version="1.0" standalone='yes'?>

<job>
  <shortName>1445628035</shortName>
  <crawlControllerState>NASCENT</crawlControllerState>
  <statusDescription>Active: NASCENT</statusDescription>
  <availableActions>
    <value>checkpoint</value>
    <value>terminate</value>
    <value>teardown</value>
  </availableActions>
  <launchCount>1</launchCount>
  <lastLaunch>2015-10-23T19:20:41.645Z</lastLaunch>
  <isProfile>false</isProfile>
  <primaryConfig>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/crawler-beans.cxml</primaryConfig>
  <primaryConfigUrl>https://localhost:8443/engine/job/1445628035/jobdir/crawler-beans.cxml</primaryConfigUrl>
  <url>https://localhost:8443/engine/job/1445628035/job/1445628035</url>
  <jobLogTail>
    <value>2015-10-23T19:20:41.645Z INFO Job launched</value>
    <value>2015-10-23T19:20:41.415Z INFO Job instantiated</value>
  </jobLogTail>
  <uriTotalsReport>
    <downloadedUriCount>0</downloadedUriCount>
    <queuedUriCount>0</queuedUriCount>
    <totalUriCount>0</totalUriCount>
    <futureUriCount>0</futureUriCount>
  </uriTotalsReport>
  <sizeTotalsReport>
    <dupByHash>0</dupByHash>
    <dupByHashCount>0</dupByHashCount>
    <notModified>0</notModified>
    <notModifiedCount>0</notModifiedCount>
    <novel>0</novel>
    <novelCount>0</novelCount>
    <total>0</total>
    <totalCount>0</totalCount>
  </sizeTotalsReport>
  <rateReport>
    <currentDocsPerSecond>0.0</currentDocsPerSecond>
    <averageDocsPerSecond>NaN</averageDocsPerSecond>
    <currentKiBPerSec>0</currentKiBPerSec>
    <averageKiBPerSec>0</averageKiBPerSec>
  </rateReport>
  <loadReport>
    <busyThreads>0</busyThreads>
    <totalThreads>0</totalThreads>
    <congestionRatio>NaN</congestionRatio>
    <averageQueueDepth>0</averageQueueDepth>
    <deepestQueueDepth>-1</deepestQueueDepth>
  </loadReport>
  <elapsedReport>
    <elapsedMilliseconds>0</elapsedMilliseconds>
    <elapsedPretty>0ms</elapsedPretty>
  </elapsedReport>
  <threadReport/>
  <frontierReport>
    <totalQueues>0</totalQueues>
    <inProcessQueues>0</inProcessQueues>
    <readyQueues>0</readyQueues>
    <snoozedQueues>0</snoozedQueues>
    <activeQueues>0</activeQueues>
    <inactiveQueues>0</inactiveQueues>
    <ineligibleQueues>0</ineligibleQueues>
    <retiredQueues>0</retiredQueues>
    <exhaustedQueues>0</exhaustedQueues>
    <lastReachedState>PAUSE</lastReachedState>
  </frontierReport>
  <crawlLogTail></crawlLogTail>
  <configFiles>
    <value>
      <key>loggerModule.crawlLogPath</key>
      <name>crawl.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/crawl.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/crawl.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>org.archive.modules.deciderules.surt.SurtPrefixedDecideRule#1259f252.surtsDumpFile</key>
      <name>surtsDumpFile</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/negative-surts.dump</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/negative-surts.dump</url>
      <editable>false</editable>
    </value>
    <value>
      <key>actionDirectory.actionDir</key>
      <name>ActionDirectory source directory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/action</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/action</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.path</key>
      <name>logs subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.nonfatalErrorsLogPath</key>
      <name>nonfatal-errors.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/nonfatal-errors.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/nonfatal-errors.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>statisticsTracker.reportsDir</key>
      <name>reports subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/reports</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/reports</url>
      <editable>false</editable>
    </value>
    <value>
      <key>warcWriter.storePaths[0]</key>
      <name>warcWriter.storePaths[0]</name>
      <path>/Applications/WAIL.app/archives</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/archives</url>
      <editable>false</editable>
    </value>
    <value>
      <key>checkpointService.checkpointsDir</key>
      <name>checkpoints subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/checkpoints</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/checkpoints</url>
      <editable>false</editable>
    </value>
    <value>
      <key>warcWriter.defaultStorePaths[0]</key>
      <name>warcs default store path</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/warcs</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/warcs</url>
      <editable>false</editable>
    </value>
    <value>
      <key>actionDirectory.doneDir</key>
      <name>ActionDirectory done directory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/actions-done</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/actions-done</url>
      <editable>false</editable>
    </value>
    <value>
      <key>crawlController.scratchDir</key>
      <name>scratch subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/scratch</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/scratch</url>
      <editable>false</editable>
    </value>
    <value>
      <key>warcWriter.directory</key>
      <name>writer base path</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.uriErrorsLogPath</key>
      <name>uri-errors.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/uri-errors.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/uri-errors.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>bdb.dir</key>
      <name>bdbmodule subdirectory</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/state</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/state</url>
      <editable>false</editable>
    </value>
    <value>
      <key>org.archive.modules.deciderules.surt.SurtPrefixedDecideRule#51cdf94.surtsDumpFile</key>
      <name>surtsDumpFile</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/surts.dump</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/surts.dump</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.progressLogPath</key>
      <name>progress-statistics.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.alertsLogPath</key>
      <name>alerts.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/alerts.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/alerts.log</url>
      <editable>false</editable>
    </value>
    <value>
      <key>loggerModule.runtimeErrorsLogPath</key>
      <name>runtime-errors.log</name>
      <path>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/runtime-errors.log</path>
      <url>https://localhost:8443/engine/job/1445628035/engine/anypath//Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/runtime-errors.log</url>
      <editable>false</editable>
    </value>
  </configFiles>
  <isLaunchInfoPartial>false</isLaunchInfoPartial>
  <isRunning>true</isRunning>
  <isLaunchable>false</isLaunchable>
  <hasApplicationContext>true</hasApplicationContext>
  <alertCount>0</alertCount>
  <checkpointFiles></checkpointFiles>
  <alertLogFilePath>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/./jobs/1445628035/20151023192043/logs/alerts.log</alertLogFilePath>
  <crawlLogFilePath>/Applications/WAIL.app/bundledApps/heritrix-3.2.0/./jobs/1445628035/20151023192043/logs/crawl.log</crawlLogFilePath>
  <reports>
    <value>
      <className>CrawlSummaryReport</className>
      <shortName>CrawlSummary</shortName>
    </value>
    <value>
      <className>SeedsReport</className>
      <shortName>Seeds</shortName>
    </value>
    <value>
      <className>HostsReport</className>
      <shortName>Hosts</shortName>
    </value>
    <value>
      <className>SourceTagsReport</className>
      <shortName>SourceTags</shortName>
    </value>
    <value>
      <className>MimetypesReport</className>
      <shortName>Mimetypes</shortName>
    </value>
    <value>
      <className>ResponseCodeReport</className>
      <shortName>ResponseCode</shortName>
    </value>
    <value>
      <className>ProcessorsReport</className>
      <shortName>Processors</shortName>
    </value>
    <value>
      <className>FrontierSummaryReport</className>
      <shortName>FrontierSummary</shortName>
    </value>
    <value>
      <className>ToeThreadsReport</className>
      <shortName>ToeThreads</shortName>
    </value>
  </reports>
  <heapReport>
    <usedBytes>40621832</usedBytes>
    <totalBytes>80216064</totalBytes>
    <maxBytes>238551040</maxBytes>
  </heapReport>
</job>

Trying to access Heritrix service at https://127.0.0.1:8443
Pseudo-Success in accessing https://127.0.0.1:8443
Trying to access Wayback service at http://localhost:8080/wayback/
Wayback is a go! 
DONE!
['2015-10-23T19:20:46.455Z INFO RUNNING 20151023192043\n']
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
No job info yet
You must run a job before stats can be shown here
['2015-10-23T19:20:46.455Z INFO RUNNING 20151023192043\n']
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
Writing path-index.txt file
Done writing path-index.txt file
CDX files generated for each WARC, merging...
Done merging CDX files, removing old source CDX files.
Sorting CDX entries
Removing unsorted temp file
Done creating sorted CDX file!
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
No job info yet
You must run a job before stats can be shown here
No job info yet
You must run a job before stats can be shown here
['2015-10-23T19:20:46.455Z INFO RUNNING 20151023192043\n']
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
/Applications/WAIL.app/bundledApps/heritrix-3.2.0/jobs/1445628035/20151023192043/logs/progress-statistics.log
machawk1 commented 8 years ago

Deleting the binary then reinstalling maintains correct functionality. Maybe this is an issue of the tomcat classpath being set. Should look at the difference in the procedure between starting Heritrix from the Basic Interface and "FIX"ing the Heritrix process.

machawk1 commented 8 years ago

This error was also verbally reported by Heather Noel Turner of MSU on OS X 10.11.1.

machawk1 commented 5 years ago

Verified as resolved per above.