Dear, evidence

I am a Bigquery very-enduser for some reporting work in my company. I am not IT savvy, but know some SQL for my reporting work

I started using VSCode+SQLTools(Bigquery Driver) for the past few weeks, and the reason is because the Web version of BQ is slow and sometimes becomes non-responsive in the Chrome Browser. With VSCode environment, the BQ work has become much much smoother than Chrome Browser (But I have my own limitation which I will share below)

However, VSCode+SQLTools(Bigquery Driver) is not officially supported by my company IT Department, thus, I have to come to you. I have 2 issues as far as I encountered.

1. In web browser, I can see a list of subfolders under the project_id_name, e.g. Queries, Data Canvases, Data Preparations, Database etc. But when I click to expand the project_id_name in VSCode Bigquery Driver, it comes with error message, saying I don't have permission to access INFORMATION_SCHEMA.SCHEMATA. I believe I was given limited access to BQ from the company IT Dept, so I could understand such limitation. (Currently I submitted the request for more access right). My Question is that, once I am given the access to SCHEMATA, would it BigQuery Driver in VSCode be able to list those subfolders (Queries, Data Preparations, Database etc?)

2. The 2nd problem is the most irritating, thus, let me explain in a few subpoints

2.1. As I can't list the subfolders as mentioned above, I realized I could use another VSCode extension (Bigquery Runner) to list the Tables of the Database. So that I could easily copy different Table-IDs to my SQL file without the need to referring back to the Web-Browser. The Bigquery Runner couldn't show other Subfolders like Queries, in which I have many customized queries saved in this subfolder Queries when I was using Web-version of BigQuery. Of course, it should be quite easy to copy-paste my queries to the local drive.

2.2. The above encounters all happened in Windows 10 environment few weeks ago, it could query anything without problem at beginning. But out of sudden (after half or one hour later), It couldn't query complicated query script (SQLTools Result blank with Spinning Cycle). Those complicated SQL script usually took > 30s in METABASE environment (the old database visualization system my company used). I restarted VSCode, it doesn't help also, But it still can query simple script, like: (select * from table where name = 'xxx' ). So in the following few days, I could only play with simple query scripts. But, out of sudden I could query very-complex script from last week though the speed is slower than Web-version and BigQuery Runner.

2.3. I still like to use your BigQuery Driver, because the output (preview) is in a much nice format than BigQuery Runner, which expand the cell too big when the string data is very long. Meanwhile, I am still expecting your BigQuery Driver could list those Subfolders in one day when I am given access to the SCHEMATA.

2.4. A New OLD Problem in MacOS. I have a Mac-Mini 2011 (MacOS Catalina 10.15.7), I used it once a while just keep my self warm with Mac environment and for fun as well. After I have installed all the packages and environment (e.g. Python, Nodejs, GCloud etc), BigQuery Driver is able to connect to the server, but I could NOT query ANY simple or complex script, the SQLTools result window keep spinning for 1-2minute. But this time, it comes with Error message says "ERROR: Error fetching records. Connection got disposed" as screenshot below.
Meanwhile, the connection is lost, and I could NOT reconnect back (There is No Error message, but Output is the same) unless I restart VSCode.

I also copied the OUTPUT from the VSCode to paste below for your reference.

Thanks in advance for your help !!!

Connection got disposed

===============================================================================

[1726104416166] INFO (ext): SQLTools is starting [1726104416183] INFO (ext): initializing language client... [1726104416183] INFO (ext): Detecting node path (if this stalls check Terminal view for the stuck session and kill it)... ns: "lc" [1726104418616] INFO (ext): Node runtime auto-detected. Using /usr/local/bin/node. ns: "lc" [1726104418618] INFO (ext): Registering client for languages [{"scheme":"sqltools"},{"language":"sql","scheme":"untitled"},{"language":"sql","scheme":"file"},{"language":"sql","scheme":"sqltools"}] ns: "lc" [1726104418623] INFO (ext): loading plugins... [1726104418624] INFO (ext): registering Formatter Plugin. plugin: { "name": "Formatter Plugin" } [1726104418625] INFO (ext): registering Connection Manager Plugin. plugin: { "name": "Connection Manager Plugin" } [1726104418625] INFO (ext): registering CodeLens Plugin. plugin: { "name": "CodeLens Plugin", "registeredLanguages": [] } [1726104418627] INFO (ext): registering Dependency Manager Plugin. plugin: { "name": "Dependency Manager Plugin", "installingDrivers": [] } [1726104418633] INFO (ext): Connection explorer changed. Will be updated. ns: "conn-man:explorer" [1726104418633] INFO (ext): registering History Manager Plugin. plugin: { "name": "History Manager Plugin" } [1726104418634] INFO (ext): registering Bookmarks Manager Plugin. plugin: { "name": "Bookmarks Manager Plugin" } [1726104418637] INFO (ext): registering Authentication Provider Plugin. plugin: { "name": "Authentication Provider Plugin", "isRegistered": false } [1726104418638] INFO (ext): registering Object Drop-in Provider (plugin). plugin: { "type": "plugin", "name": "Object Drop-in Provider", "isRegistered": false } [1726104418640] INFO (ext): SQLTools activation completed. 2473.487684 ms [1726104418647] INFO (ext): Formatter Plugin registered! [1726104418647] INFO (ext): CodeLens Plugin registered! [1726104418647] INFO (ext): Dependency Manager Plugin registered! [1726104418648] INFO (ext): Connection Manager Plugin registered! [1726104418648] INFO (ext): History Manager Plugin registered! [1726104418648] INFO (ext): Bookmarks Manager Plugin registered! [1726104418648] INFO (ext): Authentication Provider Plugin registered! [1726104418649] INFO (ext): Object Drop-in Provider (plugin) registered! [1726104418702] INFO (ext): registering BigQuery Driver for SQLTools Plugin (driver). plugin: { "extensionId": "Evidence.sqltools-bigquery-driver", "name": "BigQuery Driver for SQLTools Plugin", "type": "driver" } [1726104419033] INFO (ls): SQLTools Server started!

Using node runtime?: yes ExecPath: /usr/local/bin/node v20.17.0

[1726104419218] INFO (ext): LanguageClient ready ns: "lc" [1726104419218] INFO (ls): Initialized with node version:v20.17.0 [1726104419230] INFO (ls): REQUEST RECEIVED => ls/RegisterPlugin [1726104419234] INFO (ls): request to register plugin: "/Users/Peter/.vscode/extensions/evidence.sqltools-bigquery-driver-0.0.7/out/ls/plugin.js" [1726104419379] INFO (ls): Driver bigquery registered! place: "driver-map" ns: "ls-context" [1726104419386] INFO (ext): BigQuery Driver for SQLTools Plugin (driver) registered! [1726104420388] INFO (ext): EXECUTING COMMAND => sqltools.refreshTree [1726104420388] INFO (ext): Connection explorer changed. Will be updated. ns: "conn-man:explorer" [1726104420637] INFO (ext): Connection explorer changed. Will be updated. ns: "conn-man:explorer" [1726104445703] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104445704] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104448305] INFO (ext): EXECUTING COMMAND => sqltools.selectConnection [1726104448306] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104448307] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104448309] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104448309] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104448311] INFO (ls): REQUEST RECEIVED => connection/ConnectRequest [1726104448313] INFO (ls): Connection instance created for BigQuery. ns: "conn-manager" [1726104450333] INFO (ext): Connection explorer changed. Will be updated. ns: "conn-man:explorer" [1726104450340] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104450341] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104450381] INFO (ext): Connection explorer changed. Will be updated. ns: "conn-man:explorer" [1726104450893] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104450896] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104451021] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104451021] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104451023] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104451024] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104451025] INFO (ext): Connection explorer changed. Will be updated. ns: "conn-man:explorer" [1726104451300] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104451301] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104753125] INFO (ext): EXECUTING COMMAND => sqltools.executeQuery [1726104753127] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104753129] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104753129] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104753130] INFO (ext): Connection explorer changed. Will be updated. ns: "conn-man:explorer" [1726104753131] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104753132] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104753132] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104753134] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104754649] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726104754650] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726104754652] INFO (ls): REQUEST RECEIVED => connection/RunCommandRequest

<--- Last few GCs --->

[679:0x7ffb87000000] 517182 ms: Scavenge 2027.5 (2077.5) -> 2023.0 (2079.2) MB, 5.70 / 0.00 ms (average mu = 0.605, current mu = 0.358) allocation failure; [679:0x7ffb87000000] 517205 ms: Scavenge 2029.2 (2079.2) -> 2024.5 (2080.7) MB, 6.05 / 0.00 ms (average mu = 0.605, current mu = 0.358) allocation failure; [679:0x7ffb87000000] 517232 ms: Scavenge 2030.6 (2080.7) -> 2025.9 (2098.2) MB, 6.73 / 0.00 ms (average mu = 0.605, current mu = 0.358) allocation failure;

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory ----- Native stack trace -----

1: 0x101bf6d00 node::OOMErrorHandler(char const, v8::OOMDetails const&) [/usr/local/bin/node] 2: 0x101daf9fc v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate, char const, v8::OOMDetails const&) [/usr/local/bin/node] 3: 0x101f9f797 v8::internal::Heap::FatalProcessOutOfMemory(char const) [/usr/local/bin/node] 4: 0x101f9dff9 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node] 5: 0x101f92861 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node] 6: 0x101f932a5 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node] 7: 0x101f74bfc v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/usr/local/bin/node] 8: 0x101f68504 v8::internal::FactoryBase::NewFixedArrayWithFiller(v8::internal::Handle, int, v8::internal::Handle, v8::internal::AllocationType) [/usr/local/bin/node] 9: 0x1022b2eff v8::internal::Handle v8::internal::NameDictionary::New(v8::internal::Isolate, int, v8::internal::AllocationType, v8::internal::MinimumCapacity) [/usr/local/bin/node] 10: 0x102238895 v8::internal::JSObject::MigrateToMap(v8::internal::Isolate, v8::internal::Handle, v8::internal::Handle, int) [/usr/local/bin/node] 11: 0x10228b6d1 v8::internal::LookupIterator::ApplyTransitionToDataProperty(v8::internal::Handle) [/usr/local/bin/node] 12: 0x1022c5bae v8::internal::Object::TransitionAndWriteDataProperty(v8::internal::LookupIterator, v8::internal::Handle, v8::internal::PropertyAttributes, v8::Maybe, v8::internal::StoreOrigin) [/usr/local/bin/node] 13: 0x1023fa1b1 v8::internal::Runtime::SetObjectProperty(v8::internal::Isolate, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::StoreOrigin, v8::Maybe) [/usr/local/bin/node] 14: 0x1023fb199 v8::internal::Runtime_SetKeyedProperty(int, unsigned long, v8::internal::Isolate) [/usr/local/bin/node] 15: 0x1027c4536 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/usr/local/bin/node] 16: 0x10a8622b7 17: 0x1027def30 Builtins_ArrayForEach [/usr/local/bin/node] [Info - 09:35:36] Connection to server got closed. Server will restart. [1726104936999] ERROR (ext): ERROR: Error fetching records. Connection got disposed., {} ns: "error-handler" [1726104937335] INFO (ls): SQLTools Server started!

Using node runtime?: yes ExecPath: /usr/local/bin/node v20.17.0

[1726104937350] INFO (ls): Initialized with node version:v20.17.0 [1726105114153] INFO (ext): EXECUTING COMMAND => sqltools.showOutputChannel [1726105195707] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726105195710] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726105195714] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726105195715] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest [1726105195718] INFO (ls): REQUEST RECEIVED => connection/ConnectRequest [1726105195721] ERROR (ls): Connecting error: {"code":1000,"data":{"notification":"LS/driverNotInstalled","dontNotify":true,"args":{"driverName":"BigQuery"}}} ns: "conn-manager" [1726105195728] INFO (ext): Connection explorer changed. Will be updated. ns: "conn-man:explorer" [1726105195790] INFO (ext): EXECUTING COMMAND => sqltools.getConnections [1726105195792] INFO (ls): REQUEST RECEIVED => connection/GetConnectionsRequest

================================================================================

evidence-dev / sqltools-bigquery-driver

Error fetching records. Connection got disposed #24

Using node runtime?: yes ExecPath: /usr/local/bin/node v20.17.0

Using node runtime?: yes ExecPath: /usr/local/bin/node v20.17.0