Spotty v1.2

[apls777]

• a format of the configuration file was changed:
  • container parameters were separated from instance parameters
  • a configuration file describes a list of instances, not just one
  • added an abstraction over cloud providers. Spotty still supports only AWS, but now it's relatively easy to add any other cloud provider. As result, all specific to AWS commands were moved under the "spotty aws" command.
  • added an abstraction over volumes. Spotty still supports only EBS volumes, but now it's possible to add support for EFS or S3 volumes.
  • added the "spotty convert-config" command to convert configs to the new format
  • old configuration files are still supported, but the warning will be displayed
• deletion policies are being applied through AWS API (corresponding lambda functions were removed from the CloudFormation template). So now deletion policies can be changed in a configuration file after the instance is started.
• changes in the "spotty run" command:
  • a tmux window won't be killed once the process is exited. So now the user can see an output of the exited process. The "Ctrl+b, then x" combination of keys closes a tmux window.
  • by default, the "spotty run" command won't log script outputs. The user can use the "-l" flag to enable logging.
  • added the "-r" flag to restart a tmux session without closing it. But before using it, the running process should be stopped, because a killed "docker exec" command won't kill a spawned process automatically (see the issue here).
  • scripts can be parametrized using Mustache tags. The user can specify parameters using the "-p" flag.
• custom commands can be integrated with Spotty using Python entry points
• instances won't write logs to CloudWatch
• volumes can be restored from a custom snapshot
• added the "dry-run" flag for the "spotty start" command
• added the "debug-mode" flag for the "create-ami" command. In this case, an AMI will not be created and the user will be able to connect to the running instance.
• added the "spotty status" command that shows a current state of the instance
• added the "spotty download" command to download any project files from the instance
• added the "-l" flag for the "spotty ssh" command to list opened tmux sessions
• added the "amiId" parameter to the configuration. It can be useful to share Spotty AMI with users, so they won't need to create it.

[jagin]

Hi @apls777 ! What is the plan for the 1.2 release? I would like to start a new project with your Spotty :)

[apls777]

Hi @jagin,

I need to make some minor changes there and it's ready to go. I would say in 1-2 weeks it should be released.

I'm using this version for my current projects. It would be great if you can install it in development mode and use for your project. It's good to have somebody else to test it :). You can find an example of a new configuration file here.

[jposor]

Hey, thank you for sharing this great tool! As far as I understand, it isn't possible to use other instance types than GPU ones. Are you planning to enable other types (e.g. compute) too? Thanks!

[apls777]

Hi @jposor,

Thank you!

You can use CPU instances as well, except Nitro-based instances. But Spotty requires a GPU instance to create an AMI because it's installing NVIDIA drivers there. So once you've created an AMI, just change a type to a CPU one and start an instance.

Nitro-based instances also will be supported later, it's in the todo list. They have different device names for EBS volumes, so some changes required to mount them correctly.