psf_utils.lextab error

[JacobEFO]

Hi Ken,

I recently stumbled upon your library here, absolutely perfect for my use case of automating my simulations and reporting. However, I have yet to make it work properly.

As soon as I try to load a "psf.trn" file generated by ADE, either using:

list-psf -f psf.trn

or

from psf_utils import PSF
psf = PSF('psf.trn')

I get the following error:

WARNING: Couldn't write lextab module 'psf_utils.lextab'. [Errno 2] No such file or directory: 'C:\\Users\\jovergaard\\Anaconda3\\envs\\def_3.7\\lib\\site-packages\\psf_utils-0.4.0-py3.7.egg\\psf_utils\\lextab.py'
WARNING: Couldn't create 'psf_utils.parsetab'. [Errno 2] No such file or directory: 'C:\\Users\\jovergaard\\Anaconda3\\envs\\def_3.7\\lib\\site-packages\\psf_utils-0.4.0-py3.7.egg\\psf_utils\\parsetab.py'
Traceback (most recent call last):
  File "C:\Users\jovergaard\Anaconda3\envs\def_3.7\Scripts\list-psf-script.py", line 11, in <module>
    load_entry_point('psf-utils==0.4.0', 'console_scripts', 'list-psf')()
  File "C:\Users\jovergaard\Anaconda3\envs\def_3.7\lib\site-packages\psf_utils-0.4.0-py3.7.egg\psf_utils\list.py", line 45, in list_signals
  File "C:\Users\jovergaard\Anaconda3\envs\def_3.7\lib\site-packages\psf_utils-0.4.0-py3.7.egg\psf_utils\psf.py", line 72, in __init__
  File "C:\Users\jovergaard\Anaconda3\envs\def_3.7\lib\pathlib.py", line 1217, in read_text
    return f.read()
  File "C:\Users\jovergaard\Anaconda3\envs\def_3.7\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 498: character maps to <undefined>

I have tried this with both python 3.7 and 3.8 using Anaconda, both generating the exact same issues. I install the 'psf_utils' into my given environment with:

python setup.py install

Would you have any idea as to what goes wrong?

My Python 3.7.7 conda environment can be created with: https://pastebin.com/3jtYnEKg

Edit: To add, it makes no difference installing psf_utils through Pypi or manually from GH; same error occurs.

[KenKundert]

This issue is that the your PSF file has a file encoding other than ASCII. All my psf files are pure ASCII, but perhaps you use a different character set. The first step is to figure out which encoding the file uses. If you can run on linux you might be able to figure it out by running the file command:

> file pnoise.raw/pss.td.pss 
pnoise.raw/pss.td.pss: ASCII text

Once you figure it out, let me know, I will add it to my code. It is probably utf-8 or latin1 (ISO-8859-1). You can fix the code your self by adding the encoding to line 72 in psf_utils\psf.py:

            content = psf_filepath.read_text(encoding='utf-8', errors='replace')        

Any way, let me know what you figure out and I will fix the code accordingly.

[KenKundert]

If you are automating simulations with python, you might want to take a look at flicker-noise. I spent a fair amount of time and effort polishing the simulation script it includes to act as a good model and starting point for automated simulations.

[JacobEFO]

Right, never considered it might be my encoding. I have currently tried both latin-1 and utf-8, and it does in fact remove that error, but there are still symbols it doesn't encode properly.

Current states that � is an illegal character, telling it still doesn't decode correctly. I have tried with both replace, ignore, backslashreplace and xmlcharrefreplace for error handling.

Regarding: file psf.trn I get: psf.trn: data

But I see you have this pnoise.raw/, I have no such file/folder anywhere.

If you are automating simulations with python, you might want to take a look at flicker-noise. I spent a fair amount of time and effort polishing the simulation script it includes to act as a good model and starting point for automated simulations.

Yeah, I've briefly looked into it. Right now I am looking into invoking irun and runams with my given testbenches set up in Virtuoso, and extracting results from a mix of log files and psf files.

[KenKundert]

You can try sending me your psf.trn file if it is not too big, I can try to take a stab at determining its encoding. Your using AMS Designer, flicker-noise is more of a pure Spectre script. The issue with lextab and parsetab files is a separate issue. The parser tries to create and save tables the first time it runs to optimize performance. For some reason it cannot save those files. You might want to take a look at the path it printed and see if you can help it out. Perhaps a directory is missing or is not writeable. I have updated psf_utils on github. It now assumes 'utf-8' and adds errors='replace' to keep things going. Once things settle down, I will update pypi.

[KenKundert]

Are you sure you are creating an ASCII PSF file? psf_utils cannot read binary PSF files. If you are reading ASCII PSF files, then any errors you see should only be in the signal names, which might make it impossible to access those signals, but it should not corrupt the data.

[KenKundert]

After running some experiments, I now believe your problem is that you are trying to read a binary PSF file, which psf_utils does not support. You can use the psf command to convert your file:

psf psf.trn psf.trn.ascii
list-psf -f psf.trn.ascii

Or you can instruct the simulator to create ASCII PSF files so that no conversion is necessary.

[JacobEFO]

It seems like you're absolutely right. The issue is, that I do not have ASCII psf files. I was wondering, how psf_utils could in fact read binary psf files (which it could not).

But I realized, even though the files are called "psf.trn" and so forth, they are actually sst2. Inside ADE L, the only alternative option is psfxl files, no psf nor ascii psf.

And using either command line tools 'psf'/'psfxl' to read the psfxl file generates a header error, because it seems like headers are stored in a separate file from the one I wish to convert.

[KenKundert]

You could try using the SPECTRE_DEFAULTS environment variable:

setenv SPECTRE_DEFAULTS '--format psfascii'

That might work.

[JacobEFO]

The current issue is, that I run with AMS simulator.

Do you know, if a similar environment variable exists for AMS simulator?

[KenKundert]

It is read directly by the Spectre engine. It might work.

[JacobEFO]

Should this be done in the CIW? I am not familiar with setting environment variables in Virtuoso.

However doing that in CIW: setenv SPECTRE_DEFAULTS '--format psfascii' Generates a suspicious newline after the last single quote. If I remove quotes altogether, it complains that setenv is an undefined function.

[KenKundert]

You should have to enter it in the shell before you start virtuoso.

[JacobEFO]

My bash doesn't recognize the setenv command.

[KenKundert]

Oh, most people using Cadence use tcsh. For bash do the following:

SPECTRE_DEFAULTS='--format psfascii' virtuoso &

or

export SPECTRE_DEFAULTS='--format psfascii'
virtuoso &

[JacobEFO]

Oh, of course. Didn't consider it as a generic environment variable. Funny with tcsh, everyone I know of use bash.

Unfortunately changing the format to psfascii still seems to create a binay coded file, that is still not readable with 'psf'.

[JacobEFO]

A small update:

The results obtained from AC simulations are in true psf file formats. Where I can successfully convert from binary psf to ascii psf using the psf tool. Thereafter I can use you psf_utils to extract and plot, whatever results I want. So excellent start.

For handling the binary sst2 files, simvisdbutil seems to be able to convert the files to csv, that can then be plotted.

[KenKundert]

Thanks for the pointer to simvisdbutil. I will update the documentation to warn people about AMS Designer and SST files. Other than that, I think this issue is resolved as far as psf-utils is concerned, correct?

[JacobEFO]

My ADE L shows no option for saving as psf files, regardless of simulator. Although spectre defaults to 'psfxl' and ams to 'sst2', just fiy. So maybe the newer Virtuoso versions are slightly different in terms of output format.

Yeah, absolutely. Thank you very much for your assistance, Ken.