Style transfer - how to access/change parameters

[saddif]

Hi all,

Thanks for the great work.

I have been using the style transfer to make some nice images, however in the latest versions being able to change the style loss in advanced parameters does not seem to work any more. I get the same result regardless of the values i put into the style loss mult.

Please can you give me some guidance on how i can access them - i can't find the documentation for this?

I am using Google Colabs to run it. And this has occurred in every release of turicreate after 6, previous the V6, i can alter the style params.

This is my code

!pip install turicreate !pip uninstall -y tensorflow !pip install tensorflow-gpu import turicreate as tc tc.config.set_num_gpus(1) from google.colab import drive drive.mount('./drive') import os os.chdir('drive/My Drive/trainingdata') style = tc.load_images('style') content = tc.load_images('content')

params = { 'print_loss_breakdown': True, 'style_loss_mult': [ 1e-2, 1e-2, 1e-2, 1e-2], 'finetune_all_params': True, }

model = tc.style_transfer.create(style, content, max_iterations = 10000, _advanced_parameters=params)

This runs using the GPU and the style is transferred. But if i try to tweak the params to get more or less style transferred - i get the same result every time.

Please let me know what i am doing wrong

Thanks

[TobyRoseman]

Looking at the API docs_advanced_parameters is getting treated as part of the kwargs. Looking at the implementation for tc.style_transfer.create, _advanced_parameters will get added to the params variable. However it doesn't seem like params is ever passed through to the C++ implementation.

So yes, I agree _advanced_parameters is not getting used anymore. I don't think anything has change here since TuriCreate 6.0.

@saddif - did _advanced_parameters work in 6.0 or any version after that?

Related to #3190

[saddif]

hi, i think i mentioned this problem in - "Turicreate 6.2.2 Style Transfer parameters not working? #3190" - the last working version i have tried was V5.8.

I'll try and run some with V6.0 and onwards and report back.

Thanks again for all the hard work on turicreate!

Saddif

[saddif]

Hi Toby,

So i have retried the different versions to see which ones work with advanced parameters

params = { 'print_loss_breakdown': True, 'style_loss_mult': [ 1e-2, 1e-2, 1e-2, 1e-2], 'finetune_all_params': True, }

model = tc.style_transfer.create(style, content, max_iterations = 5000, _advanced_parameters=params)

v5.8 - runs with GPU !pip uninstall -y mxnet !pip install mxnet-cu100==1.4.0.post0

Works as expected

V6.0 runs with GPU - using mxnet (same code as above) Strange results - looks like advanced parameters are being used since the output is not like 6.2.2 etc

V6.1 runs with GPU - using mxnet (same code as above) Advanced parameters not accessed - result is same as V6.2.2 etc

V6.2 runs with GPU - !pip uninstall -y tensorflow !pip install tensorflow-gpu

Advanced parameters not accessed - result is same as V6.2.2 etc

Hope that helps you - looks like something changed in V6.0 - the results are very strange and very different to V5.8, and V6.1 onwards there is no access to advanced parameters.

Saddif

[saddif]

@TobyRoseman

Do you have an update on this issue? is there a way to access the advanced parameters in the current release?

Thanks

[TobyRoseman]

@saddif - no updates.

[TobyRoseman]

It looks like the parameters you want are still in the code. There is just no way to modify them using our API. If you want to change those values, you could just change them in the code which is installed on your computer.

In order to do this, first find the location on your computer of the file you need to change. This can be done by running: print(tc.style_transfer.style_transfer.__file__)

Then open that file and change the parameters you care about.

[saddif]

Hi Toby, Thanks for the reply (sorry about creating another issue about this)

I have tried use Turicreate V6 onwards on Google colabs when trying to use the GPU, and now it doesn't appear to work (before it was working fine - i was not able to change the parameters). Please could you help with some advice?

Google colabs has CUDA 11.2 installed - this stops turicreate from working with the GPU (even if i uninstall tensorflow and install tensorflow_gpu).

I have downgraded CUDA to 10 (i read that turicreate will only work with this?), and uninstalled tensorflow and then installed tensorflow_gpu - 1.5, 1.3 and 1.13.1. When i check for GPU using

import tensorflow as tf print(tf.test.is_gpu_available()) print(tf.config.list_physical_devices('GPU'))

i get the following result - True but also an error - the same error appears when trying to run a model

True

AttributeError Traceback (most recent call last)

in () 1 import tensorflow as tf 2 print(tf.test.is_gpu_available()) ----> 3 print(tf.config.list_physical_devices('GPU')) /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation_wrapper.py in __getattr__(self, name) 104 if name.startswith('_dw_'): 105 raise AttributeError('Accessing local variables before they are created.') --> 106 attr = getattr(self._dw_wrapped_module, name) 107 if (self._dw_warning_count < _PER_MODULE_WARNING_LIMIT and 108 name not in self._dw_deprecated_printed): AttributeError: module 'tensorflow._api.v1.config' has no attribute 'list_physical_devices' What do you think i can try next?? What are the exact CUDA, tensorflow_gpu requirements for turicreate 6.4.1? Thanks in advance! I really like turicreate, but can't afford a GPU-rig myself, hence why i am stuck with using Colabs. BTW - should i open this as a new issue? not sure what to do? Thanks

[saddif]

@TobyRoseman hi, sorry I think i posted a follow up comment with more information about the tensorflow issue into the wrong ticket - this one #3190

[saddif]

@TobyRoseman - thanks for the above explanation and to try access the parameters in the .py file directly.

Unfortunately this doesnt seem to work (i have managed to get colabs and gpu working, but the parameters still dont change)- i know it should but not sure why - these are my steps in colabs

1. Install Turicreate

2. Check the tensorflow using import tensorflow print(tensorflow.version) 2.0.4

3. !pip uninstall -y tensorflow

4. !pip install tensorflow-gpu==2.0.4

5. check location of style transfer file with print(tc.style_transfer.style_transfer.file) /usr/local/lib/python3.7/dist-packages/turicreate/toolkits/style_transfer/style_transfer.py

change the params in the above file:

params = { "batch_size": batch_size, "vgg16_content_loss_layer": 2, # conv3_3 layer "lr": 0.001, "content_loss_mult": 1.0, "style_loss_mult": [1e-1, 1e-1, 1e-1, 1e-1], # conv 1-4 layers "finetune_all_params": True, "pretrained_weights": False, "print_loss_breakdown": False, "input_shape": (256, 256), "training_content_loader_type": "stretch", "use_augmentation": False, "sequential_image_processing": False,

Only used if use_augmentaion is True

    "aug_resize": 0,
    "aug_min_object_covered": 0,
    "aug_rand_crop": 0.9,
    "aug_rand_pad": 0.9,
    "aug_rand_gray": 0.0,
    "aug_aspect_ratio": 1.25,
    "aug_hue": 0.05,
    "aug_brightness": 0.05,
    "aug_saturation": 0.05,
    "aug_contrast": 0.05,
    "aug_horizontal_flip": True,
    "aug_area_range": (0.05, 1.5),
    "aug_pca_noise": 0.0,
    "aug_max_attempts": 20,
    "aug_inter_method": 2,
    "checkpoint": False,
    "checkpoint_prefix": "style_transfer",
    "checkpoint_increment": 1000,
}

I closed the file and reopened it to check the paramters had been changed - and they had.

Then all the usual to start the learning import turicreate as tc tc.config.set_num_gpus(1) style = tc.load_images('style') content = tc.load_images('content')

model = tc.style_transfer.create(style, content, max_iterations = 5000) - i have done various combinations of this to see if i need to call advanced params e.g. definig advanced params in colabs, and adding _advanced_parameters = params to this model run.

The model runs with GPU :

Using 'image' in style_dataset as feature column and using 'image' in content_dataset as feature column Downloading https://docs-assets.developer.apple.com/turicreate/models/resnet-16.mlmodel Download completed: /var/tmp/model_cache/resnet-16.mlmodel Downloading https://docs-assets.developer.apple.com/turicreate/models/vgg16-conv1_1-4_3.mlmodel Download completed: /var/tmp/model_cache/vgg16-conv1_1-4_3.mlmodel Using a GPU to create model. WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers.

but there is no difference to setting style loss to 1e-1 or 1e-4 (the default setting)

What do you think i could try next?

btw i did not change the CUDA and Cudnn - they are the default versions in colabs i.e. 11.0, V11.0.221 and cudnn = 7.65

if i do change them to V10 and cudnn 7.4 i.e. what should work with tensor_flow GPU 2.0.4- in a fresh colabs book - then GPU is not accessed and only works with CPU. Although i dont think this is the issue.

Thanks

Saddif

[TobyRoseman]

Unfortunately this doesnt seem to work (i have managed to get colabs and gpu working, but the parameters still dont change)- i know it should but not sure why -

What makes you think it isn't working?

but there is no difference to setting style loss to 1e-1 or 1e-4 (the default setting)

What do you mean by "no difference"? Are you just doing a visual inspection of outputted images or does it actually produce the exact same image? It might be that this hyper-parameter is just not that important.

What do you think i could try next?

I'd encourage you to dig into that part of the TuriCreate codebase.

[saddif]

@TobyRoseman thanks for the reply

Yes i am doing a visual comparison between the two style loss settings - previously on Turicreate 5.8 setting it to 1e-1 for all four values would make a profound difference in the output vs 1e-4. However, with v6.4.1 there is almost zero difference (maybe a pixel change here and there) - hence why i don't think it is doing anything.

But you maybe correct - maybe it is working but with the new tensorflow-gpu there is very little change.

Anyway - i guess i will have to wait for an update that will allow me to change the style loss in a similar fashion to earlier versions.

Do you when there will be an update - or everything shifting to createML (which isn't as good IMHO).

Thanks again for your help and answering all these questions