accessing the filter files from remote data toolbox

[TayebehY]

Hi I had a question about loading the filter files which are used throughout this toolbox. Anywhere that I need to load anything using the RDT it does not respond. For example, I need to load the filter to reconstruction filterFile = 'filters_mosaic0_sv75_w1_sh2_may26primaSmall'; filterFile = 'filters_mosaic0_sv20_w1_sh2_dr0'; filterFile = 'filters_mosaic0_sv10_w1_sh2_dr0'; But none of them is available.

I would really appreciate your help. Thank you so much in advance.

[DavidBrainard]

I did a bunch of work on RDT over the last few days that I think fixed problems like this. Can you try a fresh pull of RDT and try again? If it still doesn't work, can you post the error output you get in Matlab?

[TayebehY]

Thank you so much for your prompt response. I am working on the t_primaObject and when I execute the following command the last one gives the following error.

rd = RdtClient('isetbio'); rd.crp('/resources/data/istim'); % filterFile = 'filters_mosaic0_sv75_w1_sh2_may26primaSmall'; filterFile = 'filters_mosaic0_sv20_w1_sh2_dr0'; % filterFile = 'filters_mosaic0_sv10_w1_sh2_dr0'; data = rd.readArtifact(filterFile, 'type', 'mat');

ERROR:

data = rd.readArtifact(filterFile, 'type', 'mat'); Could not find artifactId of type at remotePath <resources/data/istim> Suggestion: specify the full remotePath, not just a parent path: rdtReadArtifact( ... 'remotePath', '/foo/bar/baz') rather than rdtReadArtifact( ... 'remotePath', '/foo') Suggestion: specify the artifact file type: rdtReadArtifact( ... 'type', 'jpg') rdtReadArtifact( ... 'type', 'tiff') rdtReadArtifact( ... 'type', 'gz') etc... Error using gradleFetchArtifact (line 112) error status 1 INFO: Could not find files for the given pattern(s). To obtain bash for windows, you may download Cygwin from https://cygwin.com/install.html. Alternatively, you may download Git which bundles bash from https://git-scm.com/download/win. Either way, make sure you add the path to bash to your windows environment path, then restart Matlab.

Error in rdtReadArtifact (line 91) [localPath, pomPath, downloads] = gradleFetchArtifact(configuration.repositoryUrl, ...

Error in RdtClient/readArtifact (line 287) [data, artifact, downloads] = rdtReadArtifact(obj.configuration, ...

[DavidBrainard]

OK. Your code is working for me so we'll need to do a little debugging. Try this:

1) Open file rdtConfiguration (use "edit rdtConfiguration" at the Matlab prompt 2) Line 75 is: parser.addParameter('verbosity', 0, @isnumeric); Change this to: parser.addParameter('verbosity', 1, @isnumeric); 3) Run your rd = RdtClient('isetbio') command. It will now print out a line that tells you the configuration source file.
4) Let me know 1) what files this is, 2) what is in that file (paste the contents in your reply), 3) what is in the returned struct rd.

With that I may be able to figure out why it isn't working for you.

[TayebehY]

Thank you so much for your response. This is the response I got after executing the rd = RdtClient('isetbio') command.

rd = RdtClient('isetbio') Configuration source "config from Matlab path: C:\Users\tayo1\Desktop\Tayebeh\Stanford\isetbio\iset3d\configuration\rdt-config-isetbio.json"

rd =

repositoryName: isetbio repositoryUrl: http://brainard-archiva.psych.upenn.edu/repository/isetbio/ serverUrl: http://brainard-archiva.psych.upenn.edu/ workingRemotePath: Root username: guest password: **** verbosity: 1

And this is the information in the rd struct is that the working remote path is empty and configuration struct is as follows:

rd.configuration

ans =

struct with fields:

 acceptMediaType: 'application/json'
     cacheFolder: ''
        password: [1×0 char]
  repositoryName: 'isetbio'
   repositoryUrl: 'http://brainard-archiva.psych.upenn.edu/repository/isetbio/'
requestMediaType: 'application/json'
       serverUrl: 'http://brainard-archiva.psych.upenn.edu/'
        username: 'guest'
       verbosity: 1

Thank you so much in advance for taking the time to answer my questions.

[DavidBrainard]

That looks OK. Can you check if you have java installed on your computer. If not, install it. If it is there, check the version. On Mac/Unix go to a shell window and type "java -version". If you're on Windows I don't know how to do it but I'm sure the internet does. The RDT requires version 8 of java or later.

[TayebehY]

Thank you so much for taking the time to answer my question. Actually, that was right and now I am passed that error but I got another error at this part which is causing an error. I am using a window machine and I think that is the reason.

% We must explicitly call bash when on Windows platforms. On Mac/Linux, the % use of bash is implied through the system call. command = sprintf('%s "%s"', 'bash -c', command); [status, result] = system(command); if 0 ~= status error('FetchArtifact:BadStatus', 'error status %d (%s)', status, result) end

The error:

data = rd.readArtifact(filterFile, 'type', 'mat'); Configuration source "config from Matlab path: C:\Users\tayo1\Desktop\Tayebeh\TOOLBOX\isetbio\iset3d\configuration\rdt-config-isetbio.json" Configuration source "explicit configuration struct" DYLD_LIBRARY_PATH='' TERM=${TERM:-dumb} C:/Users/tayo1/Desktop/Tayebeh/TOOLBOX/isetbio/RemoteDataToolbox/external/gradle/gradlew --daemon --refresh-dependencies -DREPOSITORY=http://brainard-archiva.psych.upenn.edu/repository/isetbio/ -DUSERNAME=guest -DPASSWORD= -DGROUP=resources.data.istim -DID=filters_mosaic0_sv20_w1_sh2_dr0 -DVERSION=+ -DEXTENSION=mat -b C:/Users/tayo1/Desktop/Tayebeh/TOOLBOX/isetbio/RemoteDataToolbox/external/gradle/fetch.gradle fetchIt Could not find artifactId of type at remotePath <resources/data/istim> Suggestion: specify the full remotePath, not just a parent path: rdtReadArtifact( ... 'remotePath', '/foo/bar/baz') rather than rdtReadArtifact( ... 'remotePath', '/foo') Suggestion: specify the artifact file type: rdtReadArtifact( ... 'type', 'jpg') rdtReadArtifact( ... 'type', 'tiff') rdtReadArtifact( ... 'type', 'gz') etc... Error using gradleFetchArtifact (line 121) error status 2 (C:/Users/tayo1/Desktop/Tayebeh/TOOLBOX/isetbio/RemoteDataToolbox/external/gradle/gradlew: line 2: $'\r': command not found C:/Users/tayo1/Desktop/Tayebeh/TOOLBOX/isetbio/RemoteDataToolbox/external/gradle/gradlew: line 18: $'\r': command not found C:/Users/tayo1/Desktop/Tayebeh/TOOLBOX/isetbio/RemoteDataToolbox/external/gradle/gradlew: line 24: $'\r': command not found C:/Users/tayo1/Desktop/Tayebeh/TOOLBOX/isetbio/RemoteDataToolbox/external/gradle/gradlew: line 52: syntax error near unexpected token $'{\r'' C:/Users/tayo1/Desktop/Tayebeh/TOOLBOX/isetbio/RemoteDataToolbox/external/gradle/gradlew: line 52:warn () { ' )

Error in rdtReadArtifact (line 91) [localPath, pomPath, downloads] = gradleFetchArtifact(configuration.repositoryUrl, ...

Error in RdtClient/readArtifact (line 287) [data, artifact, downloads] = rdtReadArtifact(obj.configuration, ...

[DavidBrainard]

Windows is not our strong point but I think what you need to do is install the cygwin command line tools, so that windows will behave more like Linux. They are free: https://www.cygwin.com.

[TayebehY]

Thank you so much for your help. Actually I even used a Linux machine and still I get an error at this stage but this time it was error status 1. However, for that linux machine, I do not have admin access and may be the reason for this problem. I am still trying different things to figure out what is the problem. However, I would really appreciate it if you let me know if there is any other way that we can access the files.

[DavidBrainard]

I went in by hand and downloaded much of what is in the repository for this project by hand. The specific files you list above are not there, but there are some with similar names.

Here's a link to download a dropbox folder with gzipped versions of what I found on the repository. These then unzip into Mat files and you should be able to adjust the code to load these in at the point where it tries to get them from the repository. https://www.dropbox.com/sh/iilym2t9il8docw/AAAGYoeck6YyXmxGU6at5VdPa?dl=1

Presumably James has copies of the missing ones and can provide, or else can explain why they are not needed or how to generate them.

The repository also has what appear to be a set of "training" files with names im_block_X where X is an integer. There are a lot of those but we could get them if you need them. I'll defer to James on that.

[TayebehY]

Thank you so much for taking the trouble to assist me. I really appreciate it.

[DavidBrainard]

You're welcome. Let me know I if this works.