Unable to import CTF files into MTEX

[NoRoomForSquares]

What do you want to do?

I want to import CTF files into MTEX for further analysis.

What data do you have?

I have large (~4GB) .ctf files from AztecCrystal (maps were created using Aztec).

What code do you use? Please provide minimalist code with code in the following form

CS = {... 
  'notIndexed',...
  crystalSymmetry('m-3m', [2.9 2.9 2.9], 'mineral', 'Iron bcc (old)', 'color', [1 0 0])};

setMTEXpref('xAxisDirection','north');
setMTEXpref('zAxisDirection','outOfPlane');

pname = 'E:\Y_W_EBSD\CTF';

fname = [pname '\EBSD Montaged Map Data - EBSD Data Map_003 (W_Middle).ctf'];

ebsd = EBSD.load(fname,"cs",CS,"ColumnNames", {"Phase", "X", "Y", "Bands", "Error", "Euler1", "Euler2", "Euler3", "MAD", "BC", "BS"},'ctf',...
  'convertEuler2SpatialReferenceFrame');

What result do you get

I get an error that says the format CTF does not match the data, it seems to be an issue with reading in the interface from the file. The problem occurs irregardless of whether I define the interface or not. The interface described in the code is the interface in the .ctf file. What result do you expect

I would expect the code to load in the .ctf file into Matlab, ready for further analysis.

Error Message

Error using loadEBSD_ctf
EBSD format 'CTF' does not match the data
 file: 'E:/Y_W_EBSD/CTF/EBSD Montaged Map Data - EBSD Data Map_003 (W_Middle).ctf'

Error in EBSD.load (line 86)
  ebsd{k} = feval(['loadEBSD_',char(interface)],...

Error in W_Middle_Matlab (line 29)
ebsd = EBSD.load(fname,"cs",CS,"ColumnNames", {"Phase", "X", "Y", "Bands", "Error", "Euler1", "Euler2", "Euler3", "MAD", "BC", "BS"},'ctf',...

Image of the header and the columns of the .ctf file in question.

What MTEX version do you use? 5.10.2

[kilir]

Hi,

what about simply ebsd = EBSD.load(fname) or ebsd = loadEBSD_ctf(fname) ?

Cheers,

Rüdiger

[NoRoomForSquares]

I get the exact same error messages.

EBSD.load:

Error using loadEBSD_ctf
EBSD format 'CTF' does not match the data
 file: 'E:/Y_W_EBSD/CTF/EBSD Montaged Map Data - EBSD Data Map_003 (W_Middle).ctf'

Error in EBSD.load (line 86)
  ebsd{k} = feval(['loadEBSD_',char(interface)],...

Error in W_Middle_Matlab (line 29)
ebsd = EBSD.load(fname);

loadEBSD_ctf:

Error using loadEBSD_ctf
EBSD format 'CTF' does not match the data
 file: 'E:/Y_W_EBSD/CTF/EBSD Montaged Map Data - EBSD Data Map_003 (W_Middle).ctf'

Error in W_Middle_Matlab (line 29)
ebsd = loadEBSD_ctf(fname);

Cheers

[kilir]

Could you provide a part of the file? E.g. the first few 1000s lines? head -n 5000 data.ctf > sample.ctf

[NoRoomForSquares]

How exactly do I do that in MatLab?

Importdata and fileread both run out of memory.

[kilir]

Sorry, that should run in any normal shell e.g. bash. From within matlab it should be system('head -n 5000 data.ctf > sample.ctf'). Eventually on Windows OS the head program might be called differently.

[NoRoomForSquares]

https://github.com/NoRoomForSquares/mtex_test/blob/main/sample.ctf

There you go.

[kilir]

Can you try ebsd = loadEBSD_ctf(fname,'keepNaN') on the entire file?

[NoRoomForSquares]

Same issue:

Error using loadEBSD_ctf
EBSD format 'CTF' does not match the data
 file: 'E://Y_W_EBSD//CTF//W_Middle.ctf'

Error in W_Middle_Matlab (line 29)
ebsd = loadEBSD_ctf(fname, "keepNaN");

[NoRoomForSquares]

Never got the CTF import to work, had to import h5oina files instead.

[zmichels]

I just noticed that the y-coordinate column value in the dataset you linked is 0.00 for every orientation. So, spatially that dataset seems to describe a straight line (rather than a grid / map). However, the metadata in the header of the file suggests an xy-grid of [6349 x 10086]. Its a bit odd, and I wonder if the mismatch is causing some issues.