record --to-file not dumping conformant json.

[gitonthescene]

When using lsp-devtools record --to-file example.json the output seems to be formatted as Python objects rather than proper json making it hard to use 3rd party tools like jq to examine the output.

Is there any reason not to use json.dumps() here? Or if it's optional, would it make sense to make json the default?

I see this in the code, but looking at the doc it's not clear to me how to simply format the output as json. Perhaps this line should be higher up in this function?

Here's a typical line of output I'm seeing:

{'jsonrpc': '2.0', 'method': 'initialize', 'params': {'processId': 41917, 'rootPath': '/Users/XXXX/K/ngnk/lsp', 'clientInfo': {'name': 'emacs', 'version': 'GNU Emacs 29.1 (build 1, aarch64-apple-darwin21.6.0, NS appkit-2113.60 Version 12.6.6 (Build 21G646))\n of 2023-08-17'}, 'rootUri': 'file:///Users/XXXX/K/ngnk/lsp', 'capabilities': {'general': {'positionEncodings': ['utf-32', 'utf-16']}, 'workspace': {'workspaceEdit': {'documentChanges': True, 'resourceOperations': ['create', 'rename', 'delete']}, 'applyEdit': True, 'symbol': {'symbolKind': {'valueSet': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]}}, 'executeCommand': {'dynamicRegistration': False}, 'didChangeWatchedFiles': {'dynamicRegistration': True}, 'workspaceFolders': True, 'configuration': True, 'semanticTokens': {'refreshSupport': True}, 'codeLens': {'refreshSupport': True}, 'fileOperations': {'didCreate': False, 'willCreate': False, 'didRename': True, 'willRename': True, 'didDelete': False, 'willDelete': False}}, 'textDocument': {'declaration': {'dynamicRegistration': True, 'linkSupport': True}, 'definition': {'dynamicRegistration': True, 'linkSupport': True}, 'references': {'dynamicRegistration': True}, 'implementation': {'dynamicRegistration': True, 'linkSupport': True}, 'typeDefinition': {'dynamicRegistration': True, 'linkSupport': True}, 'synchronization': {'willSave': True, 'didSave': True, 'willSaveWaitUntil': True}, 'documentSymbol': {'symbolKind': {'valueSet': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]}, 'hierarchicalDocumentSymbolSupport': True}, 'formatting': {'dynamicRegistration': True}, 'rangeFormatting': {'dynamicRegistration': True}, 'onTypeFormatting': {'dynamicRegistration': True}, 'semanticTokens': {'dynamicRegistration': True, 'requests': {'range': True, 'full': True}, 'tokenModifiers': ['declaration', 'definition', 'implementation', 'readonly', 'static', 'deprecated', 'abstract', 'async', 'modification', 'documentation', 'defaultLibrary'], 'overlappingTokenSupport': True, 'multilineTokenSupport': True, 'tokenTypes': ['comment', 'keyword', 'string', 'number', 'regexp', 'operator', 'namespace', 'type', 'struct', 'class', 'interface', 'enum', 'typeParameter', 'function', 'method', 'member', 'property', 'event', 'macro', 'variable', 'parameter', 'label', 'enumConstant', 'enumMember', 'dependent', 'concept'], 'formats': ['relative']}, 'rename': {'dynamicRegistration': True, 'prepareSupport': True}, 'codeAction': {'dynamicRegistration': True, 'isPreferredSupport': True, 'codeActionLiteralSupport': {'codeActionKind': {'valueSet': ['', 'quickfix', 'refactor', 'refactor.extract', 'refactor.inline', 'refactor.rewrite', 'source', 'source.organizeImports']}}, 'resolveSupport': {'properties': ['edit', 'command']}, 'dataSupport': True}, 'completion': {'completionItem': {'snippetSupport': True, 'documentationFormat': ['markdown', 'plaintext'], 'resolveAdditionalTextEditsSupport': True, 'insertReplaceSupport': True, 'deprecatedSupport': True, 'resolveSupport': {'properties': ['documentation', 'detail', 'additionalTextEdits', 'command']}, 'insertTextModeSupport': {'valueSet': [1, 2]}}, 'contextSupport': True, 'dynamicRegistration': True}, 'signatureHelp': {'signatureInformation': {'parameterInformation': {'labelOffsetSupport': True}}, 'dynamicRegistration': True}, 'documentLink': {'dynamicRegistration': True, 'tooltipSupport': True}, 'hover': {'contentFormat': ['markdown', 'plaintext'], 'dynamicRegistration': True}, 'foldingRange': {'dynamicRegistration': True}, 'selectionRange': {'dynamicRegistration': True}, 'callHierarchy': {'dynamicRegistration': False}, 'typeHierarchy': {'dynamicRegistration': True}, 'publishDiagnostics': {'relatedInformation': True, 'tagSupport': {'valueSet': [1, 2]}, 'versionSupport': True}, 'linkedEditingRange': {'dynamicRegistration': True}}, 'window': {'workDoneProgress': True, 'showDocument': {'support': True}}}, 'initializationOptions': None, 'workDoneToken': '1'}, 'id': 232}

[alcarney]

Ah, it looks like the lsp-devtools record command isn't handling the default --to-file case correctly.

There is a potential workaround, you could try setting a format string e.g. lsp-devtools record -f "{.params}" --to-file example.json. It will have the benefit of writing out valid JSON, at the expense of filtering out all the data that does not fit the pattern.

Hopefully, I will be able to make a release containing a fix soon^TM but in testing this I've spotted a few other bugs that are worth fixing... :sweat_smile:

[gitonthescene]

Thanks for getting back so quickly. I’m not in a rush. I just wanted to flag it.

[gitonthescene]

Awesome! Thanks!

[alcarney]

No worries! lsp-devtools v0.2.2 is now out with this fix included, let me know if you run into any issues! :)

[gitonthescene]

FWIW, the sample data from the documentation still has Python object format instead of json.

I did test that it v0.2.2 does produce json though. Thanks again.

[alcarney]

FWIW, the sample data from the documentation still has Python object format instead of json

Ah, thanks for the reminder - I had forgotten 😅 it should be fixed now though!