search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
164.19k stars 29.29k forks source link

Language autodetection is bad at log file formats #166332

Closed roblourens closed 11 months ago

roblourens commented 1 year ago

I very often end up pasting large log files into a new editor to analyze them. They usually get autodetected as "ini" or "yaml". i think typical log files have a predictable format with timestamp/severity that we should be able to pick up on. Our model probably isn't trained on that at all, I'd guess.

Here's an example of a snippet that is detected as "ini"

info 01:16:03.068: Jupyter Extension: ZMQ install verified.
info 01:16:03.828: Jupyter Extension: Old kernelSpecs (created by Jupyter Extension) stored in directory /Users/roblou/Library/Jupyter/kernels/__old_vscode_kernelspecs
info 01:16:03.830: Jupyter Extension: Old kernelSpecs (created by Jupyter Extension) stored in directory /Users/roblou/Library/Jupyter/kernels/__old_vscode_kernelspecs
info 01:16:04.081: Jupyter Extension: LSP Notebooks experiment is disabled -- not using Pylance
Python used in tests is /opt/conda/bin/python
info 01:16:33.997: Jupyter Extension: Checking for server existence.
info 01:16:33.999: Jupyter Extension: Checking for server usability.
info 01:16:34.000: Jupyter Extension: Starting notebook server.
info 01:16:34.001: Jupyter Extension: Connecting to server
info 01:16:34.003: Jupyter Extension: Connecting to process server
info 01:16:34.003: Jupyter Extension: Connecting server kernel http://localhost:8888
info 01:16:34.005: Jupyter Extension: Creating server with url : http://localhost:8888
info 01:16:34.254: Jupyter Extension: Connection complete server
info 01:16:34.255: Jupyter Extension: Server started.
info 01:16:34.257: Jupyter Extension: Creating server with url : http://localhost:8888
error 01:16:34.548: Jupyter Extension: Error fetching remote kernels: FetchError: request to http://localhost:8888/api/kernels?1668474994275 failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/workspaces/vscode-jupyter/node_modules/node-fetch/lib/index.js:1491:11)
    at ClientRequest.emit (node:events:526:28)
    at Socket.socketErrorListener (node:_http_client:442:9)
    at Socket.emit (node:events:526:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
error 01:16:34.827: Jupyter Extension: UniversalRemoteKernelFinder: Failed to get kernels without cache FetchError: request to http://localhost:8888/api/kernels?1668474994275 failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/workspaces/vscode-jupyter/node_modules/node-fetch/lib/index.js:1491:11)
    at ClientRequest.emit (node:events:526:28)
    at Socket.socketErrorListener (node:_http_client:442:9)
    at Socket.emit (node:events:526:28)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
error 01:16:34.835: Jupyter Extension: SessionManager:getKernelSpecs failure:  Error: Poll (@jupyterlab/services:KernelSpecManager#specs) is disposed.
    at Poll.exports.Poll.Poll.dispose (/workspaces/vscode-jupyter/node_modules/@lumino/polling/src/poll.ts:168:23)
    at KernelSpecManager.dispose (/workspaces/vscode-jupyter/node_modules/@jupyterlab/services/src/kernelspec/manager.ts:97:21)
    at JupyterSessionManager.dispose (/workspaces/vscode-jupyter/src/kernels/jupyter/session/jupyterSessionManager.ts:109:36)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at RemoteKernelFinder.listKernelsFromConnection (/workspaces/vscode-jupyter/src/kernels/jupyter/finder/remoteKernelFinder.ts:391:21)
Running tests with options {
    "ui": "tdd",
    "color": true,
    "rootHooks": {},
    "invert": false,
    "timeout": 25000,
    "retries": 0,
    "grep": "",
    "testFilesSuffix": "*.vscode.test,*.vscode.common.test",
    "exit": true,
    "reporter": "mocha-multi-reporters",
    "reporterOptions": {
        "reporterEnabled": "spec,mocha-junit-reporter,/workspaces/vscode-jupyter/out/test/common/exitCIAfterTestReporter.js,/workspaces/vscode-jupyter/out/test/web/customReporter.js"
    }
}
Tests included test/client/api.vscode.test.js,test/common/configuration/service.vscode.test.js,test/common/misc.vscode.test.js,test/common/platform/platformService.vscode.test.js,test/common/process/decoder.vscode.test.js,test/common/variables/envVarsService.vscode.test.js,test/datascience/data-viewing/showInDataViewerPythonInterpreter.vscode.test.js,test/datascience/datascienceSurveyBanner.vscode.test.js,test/datascience/debugger.vscode.test.js,test/datascience/editor-integration/hoverProvider.vscode.test.js,test/datascience/export/exportToHTML.vscode.test.js,test/datascience/export/exportToPython.vscode.test.js,test/datascience/export/exportUtil.vscode.test.js,test/datascience/export/fileConverter.vscode.test.js,test/datascience/interactiveDebugging.vscode.test.js,test/datascience/interactiveWindow.vscode.common.test.js,test/datascience/interactiveWindow.vscode.test.js,test/datascience/interactiveWindowRemote.vscode.test.js,test/datascience/interpreters/environmentActivationService.vscode.test.js,test/datascience/ipywidgets/ipyWidgetScriptManager.vscode.common.test.js,test/datascience/jupyter/kernels/installationPrompts.vscode.test.js,test/datascience/kernel-launcher/kernelFinder.vscode.test.js,test/datascience/kernelLauncher.vscode.test.js,test/datascience/notebook/diagnosticProvider.vscode.test.js,test/datascience/notebook/executionService.mock.vscode.test.js,test/datascience/notebook/executionService.vscode.test.js,test/datascience/notebook/exportFull.vscode.test.js,test/datascience/notebook/intellisense/completion.vscode.test.js,test/datascience/notebook/intellisense/completionProvider.vscode.test.js,test/datascience/notebook/intellisense/diagnostics.vscode.test.js,test/datascience/notebook/intellisense/gotodef.vscode.test.js,test/datascience/notebook/intellisense/hover.vscode.test.js,test/datascience/notebook/intellisense/interpreterSwitch.vscode.test.js,test/datascience/notebook/intellisense/semanticTokens.vscode.test.js,test/datascience/notebook/interruptRestart.vscode.test.js,test/datascience/notebook/kernelCrashes.vscode.test.js,test/datascience/notebook/kernelEvents.vscode.common.test.js,test/datascience/notebook/kernelRefresh.vscode.test.js,test/datascience/notebook/kernelSelection.vscode.test.js,test/datascience/notebook/memory.vscode.test.js,test/datascience/notebook/nonPythonKernels.vscode.test.js,test/datascience/notebook/outputDisplayOrder.vscode.test.js,test/datascience/notebook/remoteNotebookEditor.vscode.common.test.js,test/datascience/notebook/remoteNotebookEditor.vscode.test.js,test/datascience/notebook/saving.vscode.test.js,test/datascience/notebook/thirdPartyKernels.vscode.test.js,test/datascience/notebookServerProvider.vscode.test.js,test/datascience/plotViewer/plotViewer.vscode.test.js,test/datascience/telemetry.vscode.test.js,test/datascience/variableView/variableView.vscode.test.js,test/datascience/widgets/standardWidgets.vscode.common.test.js,test/datascience/widgets/thirdpartyWidgets.vscode.common.test.js,test/debugger/common/protocolparser.vscode.test.js,test/extension.serviceRegistry.vscode.test.js,test/format/extension.format.vscode.test.js,test/platform/interpreter/interpreterPackages.node.vscode.test.js,test/standalone/notifications/pythonExtensionRestartNotification.vscode.test.js
Starting & waiting for Jupyter Extension to activate
Updating Interpreter path to /opt/conda/bin/python in workspace
info 01:16:36.524: Jupyter Extension: Active Interpreter in Python API for resource '/workspaces/vscode-jupyter/src/test/datascience' is /usr/bin/python3.9, EnvType: Unknown, EnvName: '', Version: 3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110]
info 01:16:36.524: Jupyter Extension: Active Interpreter in Python API for resource '/workspaces/vscode-jupyter/src/test/datascience' is /usr/bin/python3.9, EnvType: Unknown, EnvName: '', Version: 3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110]
warn 01:16:36.551: Jupyter Extension: No interpreter with path /usr/bin/python3.9 found in Python API, will convert Uri path to string as Id file:///usr/bin/python3.9
Jupyter not started and set to local
Time taken to activate the extension: 632.906982421875 ms
Time taken to activate the extension: 632.946ms
Initialize Exit Reporter for Mocha (PVSC).
(node:14656) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 fail listeners added to [Runner]. Use emitter.setMaxListeners() to increase limit
(Use `node --trace-warnings ...` to show where the warning was created)
(node:14656) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 pass listeners added to [Runner]. Use emitter.setMaxListeners() to increase limit
Start Exit Reporter for Mocha.
Started tests
TylerLeonhardt commented 11 months ago

linking to mega thread of bad guesses: https://github.com/microsoft/vscode/issues/131450