determined-ai / determined-examples

Example ML projects that use the Determined library.

https://github.com/determined-ai/determined

Apache License 2.0

14 stars 1 forks source link

deep-learning distributed-training hyperparameter-tuning keras machine-learning ml-infrastructure pytorch tensorflow

readme

Determined As-is Examples

This repository contains a variety of Determined examples that are not actively maintained by the Determined team.

Tutorials

Example	Dataset	Framework
fashion_mnist_tf_keras	Fashion MNIST	TensorFlow (tf.keras)

Blog posts

Example	Description
Activation Memory: Part 2	Measuring activation memory in PyTorch.
LLM Finetuning	Finetuning TinyLlama-1.1B on Text-to-SQL.
LLM Finetuning 2	Finetuning Mistral-7B on Text-to-SQL using LoRA and DeepSpeed.
LLM Finetuning 3	Finetuning Gemma-2B using DPO.
LoRA Parameters	Finding the best LoRA parameters.
Python SDK demo	Example usage of the Determined Python SDK to run and administer experiments.
Tensor Parallelism	Profiling tensor parallelism in PyTorch.

Computer Vision

Example	Dataset	Framework
cifar10_pytorch	CIFAR-10	PyTorch
cifar10_pytorch_inference	CIFAR-10	PyTorch
cifar10_tf_keras	CIFAR-10	TensorFlow (tf.keras)
fasterrcnn_coco_pytorch	Penn-Fudan Dataset	PyTorch
mmdetection	COCO	PyTorch
detr_coco_pytorch	COCO	PyTorch
deformabledetr_coco_pytorch	COCO	PyTorch
iris_tf_keras	Iris Dataset	TensorFlow (tf.keras)
unets_tf_keras	Oxford-IIIT Pet Dataset	TensorFlow (tf.keras)
efficientdet_pytorch	COCO	PyTorch
byol_pytorch	CIFAR-10 / STL-10 / ImageNet	PyTorch
deepspeed_cifar10_cpu_offloading	CIFAR-10	PyTorch (DeepSpeed)

Natural Language Processing (NLP)

Example	Dataset	Framework
albert_squad_pytorch	SQuAD	PyTorch
bert_glue_pytorch	GLUE	PyTorch
word_language_model	WikiText-2	PyTorch

HP Search Benchmarks

Example	Dataset	Framework
darts_cifar10_pytorch	CIFAR-10	PyTorch
darts_penntreebank_pytorch	Penn Treebank Dataset	PyTorch

Neural Architecture Search (NAS)

Example	Dataset	Framework
gaea_pytorch	DARTS	PyTorch

Meta Learning

Example	Dataset	Framework
protonet_omniglot_pytorch	Omniglot	PyTorch

Generative Adversarial Networks (GAN)

Example	Dataset	Framework
dc_gan_tf_keras	MNIST	TensorFlow (tf.keras)
gan_mnist_pytorch	MNIST	PyTorch
deepspeed_dcgan	MNIST / CIFAR-10	PyTorch (DeepSpeed)
pix2pix_tf_keras	pix2pix	TensorFlow (tf.keras)
cyclegan	monet2photo	PyTorch

Custom Reducers

Example	Dataset	Framework
custom_reducers_mnist_pytorch	MNIST	PyTorch

HP Search Constraints

Example	Dataset	Framework
hp_constraints_mnist_pytorch	MNIST	PyTorch

Custom Search Method

Example	Dataset	Framework
asha_search_method	MNIST	PyTorch

Fully Sharded Data Parallel

Example	Framework
minimal_fsdp	PyTorch