search

aws / aws-toolkit-vscode

Amazon Q, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Terminal, AWS resources
https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.amazon-q-vscode
Apache License 2.0
1.51k stars 440 forks source link

AWS Glue Support #3272

Open ams1 opened 1 year ago

ams1 commented 1 year ago

Great tool! Thank you for open sourcing it.

Problem

As I (and probaly many others) live in vscode (python, terraform repos etc.) and interact with AWS Glue Jobs, it would be great to have a minimal (nicer than aws cli) UI for interacting with Glue jobs (without context switching).

Expected behavior

Are there any short-mid term plans for Glue Support? -> so I know if I start to implement something 😃

Thank you again

justinmk3 commented 1 year ago

any short-mid term plans for Glue Support? -> so I know if I start to implement something

Nothing implemented currently. A PR would be welcome! See https://github.com/aws/aws-toolkit-vscode/pull/2919 for a starting point about how to add basic support for a service to AWS Explorer.

ams1 commented 1 year ago

@justinmk3 thanks for the hint.

I started making some progress on this - got the jobs listed in the explorer (brief victory moment: yeey!) -, BUT...

...as I also want to implement a command to quickly retreive/show the CloudWatchLog for (ex.) the latest run of a glue job, I've "hit" the following...

...QUESTION: from a new command, what would be the most elegant (DRY) way to show as a vscode doc the CloudWatchLog for a given region, logGroupName and a given logStreamName (I have these 3 inputs from the glue job run)?

Something like: right click on job, select ~"Last run driver log" and you should see a new vscode doc with the CloudWatchLog.

I was thinking there is a function/command for this but viewLogStream does not accept logStreamName to be passed - instead it prompts you to select it using the picker.

https://github.com/aws/aws-toolkit-vscode/blob/3a22329dbc6d1eb819a65ea6ef84599f24d2cb48/src/cloudWatchLogs/commands/viewLogStream.ts#L30-L48

Not sure if here is the best place to ask these questions (if it's not, sorry -> please advise where to ask these questions).

So far it was a great ride, thank you for that!

P.S.: glue also references s3 objects -> given that the script of the glue job is on S3, something like an "open glue script" command would also make sense - I haven't looked at how to do that yet

justinmk3 commented 1 year ago

there is a function/command for this but viewLogStream does not accept logStreamName to be passed - instead it prompts you to select it using the picker.

We have a feature/cwl branch which refactors some of the Cloudwatch-related code. It might have an improvement there. We'll merge that branch soonish (in a month or so), so you might want to leave the "Last run driver log" feature as a TODO until then (though it would be a great feature)!

Alternatively, if it's not much work, it's also fine to add a viewLogStream variant or expand its signature.

Not sure if here is the best place to ask these questions (if it's not, sorry -> please advise where to ask these questions).

This is a fine and good place :) Maybe we should enable "Discussions" too.

ams1 commented 1 year ago

@justinmk3 below is just the listing of the glue jobs in explorer - I'll soon add more features

https://github.com/aws/aws-toolkit-vscode/compare/master...ams1:aws-toolkit-vscode:feature/glue_service_1?diff=unified