search

pynbody / tangos

The Agile Numerical Galaxy Organisation System
BSD 3-Clause "New" or "Revised" License
19 stars 13 forks source link

NameError: No providing class for property #133

Open Mirror-Knight opened 3 years ago

Mirror-Knight commented 3 years ago

I'm a beginner and I'm trying to write some simple properties to calculate for the tutorial data. I am running into the following error when I define a property that requires particle data

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/core/halo.py", line 145, in calculate
    (value,), description = calculation.values_sanitized_and_description([self], Session.object_session(self))
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/live_calculation/__init__.py", line 131, in values_sanitized_and_description
    values, desc = self.values_and_description(halos)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/live_calculation/__init__.py", line 405, in values_and_description
    calculator, results = self._evaluate_function(halos, input_descriptions, input_values)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/live_calculation/__init__.py", line 420, in _evaluate_function
    calculator = properties.providing_class(self.name())(sim, *input_descriptions)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/properties/__init__.py", line 405, in providing_class
    raise NameError("No providing class for property " + property_name)
NameError: No providing class for property mypropertyp

The property I'm testing is the property provided in the example (velocity dispersion). I named it "mypropertyp" since I have another "myproperty" that just returns 42 (another example provided in the tutorial docs)

from tangos.properties.pynbody import PynbodyPropertyCalculation
import numpy as np

class ExampleHaloProperty(PynbodyPropertyCalculation):
    names = "mypropertyp"

    def calculate(self, particle_data, existing_properties):
        return np.std(particle_data['vel'])

In this case my code is simply

import tangos
import testproperty2

k = tangos.get_halo("tutorial_changa_blackholes/%384/halo_4")
print(k.calculate("mypropertyp()")

This issue does not arise when pynbody particle data is not called. I have also tried adding my testproperty2.py to tangos by setting the environment variable TANGOS_PROPERTY_MODULES but that has not worked for me. Also when I tried to run the following

tangos write  --for tutorial_changa_blackholes

I would still get the same NameError. I am running tangos on a remote server (specifically the McMaster Astro server).

Thanks

apontzen commented 3 years ago

This is possibly not very well explained in the current documentation but live calculations, such as the one you are trying to do, are not allowed to use particle data. Only the 'batch'-type database updates can access particle data. You should find that you can write mypropertyp into the database using tangos write mypropertyp

Mirror-Knight commented 3 years ago

I tried that and I'm still getting the same error. Specifically I am seeing the following

(base) [luh60@astro ~]$ export TANGOS_PROPERTY_MODULES=testproperty
(base) [luh60@astro ~]$ tangos write mypropertyp
2021-01-28 21:51:58,907 : Processing <TimeStep 'tutorial_gadget/snapshot_010' z=6.07 t=0.94 Gyr>
Traceback (most recent call last):
  File "/home/luh60/anaconda3/bin/tangos", line 8, in <module>
    sys.exit(main())
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/scripts/__init__.py", line 49, in main
    args.func(args)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/tools/__init__.py", line 54, in run
    parallel_tasks.launch(obj.run_calculation_loop, 2, [])
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/parallel_tasks/__init__.py", line 56, in launch
    function(*args)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/tools/property_writer.py", line 514, in run_calculation_loop
    self.run_timestep_calculation(f_obj)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/tools/property_writer.py", line 458, in run_timestep_calculation
    self._property_calculator_instances = properties.instantiate_classes(db_timestep.simulation, self.options.properties)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/properties/__init__.py", line 450, in instantiate_classes
    instances.append(providing_class(property_identifier, handler_class, silent_fail)(simulation))
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/properties/__init__.py", line 405, in providing_class
    raise NameError("No providing class for property " + property_name)
NameError: No providing class for property mypropertyp
apontzen commented 3 years ago

This must be a different problem I think. Do you see mypropertyp if you do tangos list-possible-properties?

Mirror-Knight commented 3 years ago

I get the following

(base) [luh60@astro ~]$ tangos list-possible-properties
                          name |     handler     | property class
-------------------------------+-----------------+-------------------------------------------------------------
           BH_central_distance | PynbodyInputHan | tangos.properties.pynbody.BH.BH
             BH_central_offset | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                       BH_mass | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                               | EagleLikeInputH | tangos.properties.pynbody.eagle.EagleBH
                       BH_mass | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                               | EagleLikeInputH | tangos.properties.pynbody.eagle.EagleBH
                       BH_mdot | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                               | EagleLikeInputH | tangos.properties.pynbody.eagle.EagleBH
                       BH_mdot | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                               | EagleLikeInputH | tangos.properties.pynbody.eagle.EagleBH
                   BH_mdot_ave | PynbodyInputHan | tangos.properties.pynbody.BH.BH
             BH_mdot_histogram |   HandlerBase   | tangos.properties.pynbody.BH.BHAccHistogram
         BH_mdot_histogram_all | PynbodyInputHan | tangos.properties.pynbody.BH.BHAccHistogramMerged
                   BH_mdot_std | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                           NDM |       live      | tangos.properties.intrinsic.IntrinsicProperties
                          NGas |       live      | tangos.properties.intrinsic.IntrinsicProperties
                         NStar |       live      | tangos.properties.intrinsic.IntrinsicProperties
                    SFR_100Myr | PynbodyInputHan | tangos.properties.pynbody.SF.StarForm
                     SFR_10Myr | PynbodyInputHan | tangos.properties.pynbody.SF.StarForm
                 SFR_histogram | PynbodyInputHan | tangos.properties.pynbody.SF.StarFormHistogram
                             a |       live      | tangos.properties.intrinsic.IntrinsicProperties
                            at |       live      | tangos.properties.live_profiles.AtPosition
                            bh |       live      | tangos.properties.pynbody.BH.BHGal
                      bhcenter |       live      | tangos.properties.pynbody.BH.BHCentral
        contamination_fraction | PynbodyInputHan | tangos.properties.pynbody.zoom.Contamination
                          dbid |       live      | tangos.properties.intrinsic.IntrinsicProperties
            dm_density_profile | PynbodyInputHan | tangos.properties.pynbody.profile.HaloDensityProfile
                               |  YtInputHandler | tangos.properties.yt.HaloDensityProfile
            dm_density_profile | PynbodyInputHan | tangos.properties.pynbody.profile.HaloDensityProfile
                               |  YtInputHandler | tangos.properties.yt.HaloDensityProfile
               dm_mass_profile | PynbodyInputHan | tangos.properties.pynbody.profile.HaloDensityProfile
                               |  YtInputHandler | tangos.properties.yt.HaloDensityProfile
               dm_mass_profile | PynbodyInputHan | tangos.properties.pynbody.profile.HaloDensityProfile
                               |  YtInputHandler | tangos.properties.yt.HaloDensityProfile
                finder_dm_mass | PynbodyInputHan | tangos.properties.pynbody.mass.MassBreakdown
               finder_gas_mass | PynbodyInputHan | tangos.properties.pynbody.mass.MassBreakdown
                     finder_id |       live      | tangos.properties.intrinsic.IntrinsicProperties
                   finder_mass | PynbodyInputHan | tangos.properties.pynbody.mass.Masses
              finder_star_mass | PynbodyInputHan | tangos.properties.pynbody.mass.MassBreakdown
           gas_density_profile | PynbodyInputHan | tangos.properties.pynbody.profile.BaryonicHaloDensityProfile
                       gas_map | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
                gas_map_faceon | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
                gas_map_sideon | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
              gas_mass_profile | PynbodyInputHan | tangos.properties.pynbody.profile.BaryonicHaloDensityProfile
                   halo_number |       live      | tangos.properties.intrinsic.IntrinsicProperties
                           max |       live      | tangos.properties.live_profiles.MaxMinProperty
                    max_radius | PynbodyInputHan | tangos.properties.pynbody.centring.CentreAndRadius
           max_radius_comoving |       live      | tangos.properties.pynbody.centring.CentreAndRadiusComoving
                      mean_rho | PynbodyInputHan | tangos.properties.pynbody.gas.MeanGasProperties
                     mean_temp | PynbodyInputHan | tangos.properties.pynbody.gas.MeanGasProperties
                           min |       live      | tangos.properties.live_profiles.MaxMinProperty
                          path |       live      | tangos.properties.intrinsic.IntrinsicProperties
                        posmax |       live      | tangos.properties.live_profiles.MaxMinProperty
                        posmin |       live      | tangos.properties.live_profiles.MaxMinProperty
                 shrink_center | PynbodyInputHan | tangos.properties.pynbody.centring.CentreAndRadius
        shrink_center_comoving |       live      | tangos.properties.pynbody.centring.CentreAndRadiusComoving
          star_density_profile | PynbodyInputHan | tangos.properties.pynbody.profile.BaryonicHaloDensityProfile
             star_mass_profile | PynbodyInputHan | tangos.properties.pynbody.profile.BaryonicHaloDensityProfile
                     step_path |       live      | tangos.properties.intrinsic.IntrinsicProperties
                             t |       live      | tangos.properties.intrinsic.IntrinsicProperties
                          type |       live      | tangos.properties.intrinsic.IntrinsicProperties
                     uvi_image | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
              uvi_image_faceon | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
              uvi_image_sideon | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
                             z |       live      | tangos.properties.intrinsic.IntrinsicProperties

I am having issues with tangos write - I still get the NameError even for properties that do not require pynbody (In this case I'm trying to add myproperty, which was the very simple property that returns 42. This works only if I put import testproperty at the beginning of my python code.

apontzen commented 3 years ago

Yes, this confirms that tangos isn't importing your module. That's why you can't get tangos write to work either. Is the module definitely somewhere python can see it, eg in the folder you are executing from, or on your PYTHONPATH ?

Mirror-Knight commented 3 years ago

Yes I have the file testproperty.py in my home directory which contains myproperty. In my home directory I run tangos write myproperty

Is tangos having issues finding myproperty somehow?

apontzen commented 3 years ago

Yes. To be sure you could try putting a print statement in myproperty.py so you can easily see if/when it gets found. I don't think tangos is finding it.

Mirror-Knight commented 3 years ago

I found the issue with myproperty - I had forgotten to set the environment variable. myproperty can be written and it shows up when I run tangos list-possible-properties.

Unfortunately I am still having issue with mypropertyp - the one that calculates velocity dispersion and requires particle data.

apontzen commented 3 years ago

Are you saying tangos list-possible-properties shows myproperty but not mypropertyp? If that's the case, somehow your class for mypropertyp is just not showing up to tangos.

Perhaps post the full code in your module, hopefully I can spot the issue.

Mirror-Knight commented 3 years ago

Ive renamed mypropertyp to veldisp but the issue is that if I try to run tangos write veldisp I get the NameError. Below is what I get

(base) [luh60@astro ~]$ export TANGOS_PROPERTY_MODULES=veldisp
(base) [luh60@astro ~]$ tangos write veldisp
2021-02-01 15:43:50,199 : Processing <TimeStep 'tutorial_gadget/snapshot_010' z=6.07 t=0.94 Gyr>
Traceback (most recent call last):
  File "/home/luh60/anaconda3/bin/tangos", line 8, in <module>
    sys.exit(main())
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/scripts/__init__.py", line 49, in main
    args.func(args)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/tools/__init__.py", line 54, in run
    parallel_tasks.launch(obj.run_calculation_loop, 2, [])
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/parallel_tasks/__init__.py", line 56, in launch
    function(*args)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/tools/property_writer.py", line 514, in run_calculation_loop
    self.run_timestep_calculation(f_obj)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/tools/property_writer.py", line 458, in run_timestep_calculation
    self._property_calculator_instances = properties.instantiate_classes(db_timestep.simulation, self.options.properties)
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/properties/__init__.py", line 450, in instantiate_classes
    instances.append(providing_class(property_identifier, handler_class, silent_fail)(simulation))
  File "/home/luh60/anaconda3/lib/python3.8/site-packages/tangos/properties/__init__.py", line 405, in providing_class
    raise NameError("No providing class for property " + property_name)
NameError: No providing class for property veldisp

Here is what I had a little earlier - myproperty was successfully written in for tutorial_changa and it shows up in tangos list-possible-properties

(base) [luh60@astro ~]$ export TANGOS_PROPERTY_MODULES=myproperty
(base) [luh60@astro ~]$ tangos write myproperty --for tutorial_changa
2021-02-01 15:42:33,509 : Processing <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000384' z=4.57 t=1.31 Gyr>
2021-02-01 15:42:33,626 : Gathering existing properties for all halos in timestep <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000384' z=4.57 t=1.31 Gyr>
2021-02-01 15:42:34,609 : Successfully gathered existing properties; calculating halo properties now...
2021-02-01 15:42:34,609 :   362 halos to consider; 1 property calculations for each of them
2021-02-01 15:42:34,656 : Done with <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000384' z=4.57 t=1.31 Gyr>
2021-02-01 15:42:34,677 :             Succeeded: 362 property calculations
2021-02-01 15:42:34,677 :               Errored: 0 property calculations
2021-02-01 15:42:34,677 :   Errored during load: 0 property calculations
2021-02-01 15:42:34,678 :        Already exists: 0 property calculations
2021-02-01 15:42:34,678 : Missing pre-requisite: 0 property calculations
2021-02-01 15:42:34,678 : Processing <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000448' z=4.03 t=1.53 Gyr>
2021-02-01 15:42:34,678 : Gathering existing properties for all halos in timestep <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000448' z=4.03 t=1.53 Gyr>
2021-02-01 15:42:35,680 : Successfully gathered existing properties; calculating halo properties now...
2021-02-01 15:42:35,680 :   408 halos to consider; 1 property calculations for each of them
2021-02-01 15:42:35,727 : Done with <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000448' z=4.03 t=1.53 Gyr>
2021-02-01 15:42:35,748 :             Succeeded: 408 property calculations
2021-02-01 15:42:35,748 :               Errored: 0 property calculations
2021-02-01 15:42:35,748 :   Errored during load: 0 property calculations
2021-02-01 15:42:35,748 :        Already exists: 0 property calculations
2021-02-01 15:42:35,748 : Missing pre-requisite: 0 property calculations
2021-02-01 15:42:35,749 : Processing <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000640' z=2.96 t=2.17 Gyr>
2021-02-01 15:42:35,749 : Gathering existing properties for all halos in timestep <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000640' z=2.96 t=2.17 Gyr>
2021-02-01 15:42:37,077 : Successfully gathered existing properties; calculating halo properties now...
2021-02-01 15:42:37,077 :   545 halos to consider; 1 property calculations for each of them
2021-02-01 15:42:37,157 : Done with <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000640' z=2.96 t=2.17 Gyr>
2021-02-01 15:42:37,178 :             Succeeded: 545 property calculations
2021-02-01 15:42:37,178 :               Errored: 0 property calculations
2021-02-01 15:42:37,178 :   Errored during load: 0 property calculations
2021-02-01 15:42:37,178 :        Already exists: 0 property calculations
2021-02-01 15:42:37,178 : Missing pre-requisite: 0 property calculations
2021-02-01 15:42:37,178 : Processing <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000832' z=2.32 t=2.82 Gyr>
2021-02-01 15:42:37,179 : Gathering existing properties for all halos in timestep <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000832' z=2.32 t=2.82 Gyr>
2021-02-01 15:42:38,778 : Successfully gathered existing properties; calculating halo properties now...
2021-02-01 15:42:38,778 :   666 halos to consider; 1 property calculations for each of them
2021-02-01 15:42:38,836 : Done with <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000832' z=2.32 t=2.82 Gyr>
2021-02-01 15:42:38,860 :             Succeeded: 666 property calculations
2021-02-01 15:42:38,860 :               Errored: 0 property calculations
2021-02-01 15:42:38,860 :   Errored during load: 0 property calculations
2021-02-01 15:42:38,860 :        Already exists: 0 property calculations
2021-02-01 15:42:38,860 : Missing pre-requisite: 0 property calculations
2021-02-01 15:42:38,860 : Processing <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000960' z=2.02 t=3.25 Gyr>
2021-02-01 15:42:38,861 : Gathering existing properties for all halos in timestep <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000960' z=2.02 t=3.25 Gyr>
2021-02-01 15:42:40,626 : Successfully gathered existing properties; calculating halo properties now...
2021-02-01 15:42:40,626 :   719 halos to consider; 1 property calculations for each of them
2021-02-01 15:42:40,707 : Done with <TimeStep 'tutorial_changa/pioneer50h128.1536gst1.bwK1.000960' z=2.02 t=3.25 Gyr>
2021-02-01 15:42:40,734 :             Succeeded: 719 property calculations
2021-02-01 15:42:40,734 :               Errored: 0 property calculations
2021-02-01 15:42:40,734 :   Errored during load: 0 property calculations
2021-02-01 15:42:40,734 :        Already exists: 0 property calculations
2021-02-01 15:42:40,734 : Missing pre-requisite: 0 property calculations
2021-02-01 15:42:40,734 : Attempting to commit 2700 halo properties...
2021-02-01 15:42:46,016 : 2700 properties were committed
2021-02-01 15:42:46,019 : CUMULATIVE RUNNING TIMES (just this node)
2021-02-01 15:42:46,020 :  ExampleHaloProperty 0.0s | 100.0%
(base) [luh60@astro ~]$ tangos list-possible-properties
                          name |     handler     | property class
-------------------------------+-----------------+-------------------------------------------------------------
           BH_central_distance | PynbodyInputHan | tangos.properties.pynbody.BH.BH
             BH_central_offset | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                       BH_mass | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                               | EagleLikeInputH | tangos.properties.pynbody.eagle.EagleBH
                       BH_mass | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                               | EagleLikeInputH | tangos.properties.pynbody.eagle.EagleBH
                       BH_mdot | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                               | EagleLikeInputH | tangos.properties.pynbody.eagle.EagleBH
                       BH_mdot | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                               | EagleLikeInputH | tangos.properties.pynbody.eagle.EagleBH
                   BH_mdot_ave | PynbodyInputHan | tangos.properties.pynbody.BH.BH
             BH_mdot_histogram |   HandlerBase   | tangos.properties.pynbody.BH.BHAccHistogram
         BH_mdot_histogram_all | PynbodyInputHan | tangos.properties.pynbody.BH.BHAccHistogramMerged
                   BH_mdot_std | PynbodyInputHan | tangos.properties.pynbody.BH.BH
                           NDM |       live      | tangos.properties.intrinsic.IntrinsicProperties
                          NGas |       live      | tangos.properties.intrinsic.IntrinsicProperties
                         NStar |       live      | tangos.properties.intrinsic.IntrinsicProperties
                    SFR_100Myr | PynbodyInputHan | tangos.properties.pynbody.SF.StarForm
                     SFR_10Myr | PynbodyInputHan | tangos.properties.pynbody.SF.StarForm
                 SFR_histogram | PynbodyInputHan | tangos.properties.pynbody.SF.StarFormHistogram
                             a |       live      | tangos.properties.intrinsic.IntrinsicProperties
                            at |       live      | tangos.properties.live_profiles.AtPosition
                            bh |       live      | tangos.properties.pynbody.BH.BHGal
                      bhcenter |       live      | tangos.properties.pynbody.BH.BHCentral
        contamination_fraction | PynbodyInputHan | tangos.properties.pynbody.zoom.Contamination
                          dbid |       live      | tangos.properties.intrinsic.IntrinsicProperties
            dm_density_profile | PynbodyInputHan | tangos.properties.pynbody.profile.HaloDensityProfile
                               |  YtInputHandler | tangos.properties.yt.HaloDensityProfile
            dm_density_profile | PynbodyInputHan | tangos.properties.pynbody.profile.HaloDensityProfile
                               |  YtInputHandler | tangos.properties.yt.HaloDensityProfile
               dm_mass_profile | PynbodyInputHan | tangos.properties.pynbody.profile.HaloDensityProfile
                               |  YtInputHandler | tangos.properties.yt.HaloDensityProfile
               dm_mass_profile | PynbodyInputHan | tangos.properties.pynbody.profile.HaloDensityProfile
                               |  YtInputHandler | tangos.properties.yt.HaloDensityProfile
                finder_dm_mass | PynbodyInputHan | tangos.properties.pynbody.mass.MassBreakdown
               finder_gas_mass | PynbodyInputHan | tangos.properties.pynbody.mass.MassBreakdown
                     finder_id |       live      | tangos.properties.intrinsic.IntrinsicProperties
                   finder_mass | PynbodyInputHan | tangos.properties.pynbody.mass.Masses
              finder_star_mass | PynbodyInputHan | tangos.properties.pynbody.mass.MassBreakdown
           gas_density_profile | PynbodyInputHan | tangos.properties.pynbody.profile.BaryonicHaloDensityProfile
                       gas_map | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
                gas_map_faceon | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
                gas_map_sideon | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
              gas_mass_profile | PynbodyInputHan | tangos.properties.pynbody.profile.BaryonicHaloDensityProfile
                   halo_number |       live      | tangos.properties.intrinsic.IntrinsicProperties
                           max |       live      | tangos.properties.live_profiles.MaxMinProperty
                    max_radius | PynbodyInputHan | tangos.properties.pynbody.centring.CentreAndRadius
           max_radius_comoving |       live      | tangos.properties.pynbody.centring.CentreAndRadiusComoving
                      mean_rho | PynbodyInputHan | tangos.properties.pynbody.gas.MeanGasProperties
                     mean_temp | PynbodyInputHan | tangos.properties.pynbody.gas.MeanGasProperties
                           min |       live      | tangos.properties.live_profiles.MaxMinProperty
                    myproperty |       live      | myproperty.ExampleHaloProperty
                          path |       live      | tangos.properties.intrinsic.IntrinsicProperties
                        posmax |       live      | tangos.properties.live_profiles.MaxMinProperty
                        posmin |       live      | tangos.properties.live_profiles.MaxMinProperty
                 shrink_center | PynbodyInputHan | tangos.properties.pynbody.centring.CentreAndRadius
        shrink_center_comoving |       live      | tangos.properties.pynbody.centring.CentreAndRadiusComoving
          star_density_profile | PynbodyInputHan | tangos.properties.pynbody.profile.BaryonicHaloDensityProfile
             star_mass_profile | PynbodyInputHan | tangos.properties.pynbody.profile.BaryonicHaloDensityProfile
                     step_path |       live      | tangos.properties.intrinsic.IntrinsicProperties
                             t |       live      | tangos.properties.intrinsic.IntrinsicProperties
                          type |       live      | tangos.properties.intrinsic.IntrinsicProperties
                     uvi_image | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
              uvi_image_faceon | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
              uvi_image_sideon | PynbodyInputHan | tangos.properties.pynbody.images.BaryonicImages
                             z |       live      | tangos.properties.intrinsic.IntrinsicProperties
apontzen commented 3 years ago

I'm afraid I don't understand that test and what you were hoping it to show. Please just post the code. If tangos is definitely importing the module but can't see the property then I can probably spot the problem.

Mirror-Knight commented 3 years ago

Below is the file veldisp.py - this produces the NameError when I try to either run tangos write veldisp

from tangos.properties.pynbody import PynbodyPropertyCalculation
import numpy as np

class ExampleHaloProperty(PynbodyPropertyCalculation):
    names = "veldisp"

    def calculate(self, particle_data, existing_properties):
        return np.std(particle_data['vel'])
apontzen commented 3 years ago

Thanks! I don't see any problem with this and I was able to make it work:

$ cat > veldisp.py

from tangos.properties.pynbody import PynbodyPropertyCalculation
import numpy as np

class ExampleHaloProperty(PynbodyPropertyCalculation):
    names = "veldisp"

    def calculate(self, particle_data, existing_properties):
        return np.std(particle_data['vel'])

$ export TANGOS_PROPERTY_MODULES=veldisp

$ tangos list-possible-properties | grep veldisp

                       veldisp | PynbodyInputHan | veldisp.ExampleHaloProperty

This confirms tangos can see the property. So I am not really sure what is going on in your case -- are you certain the veldisp.py module is being found on your path? To check this, put a print statement at module level so you can see if/when it is imported.

apontzen commented 3 years ago

@Mirror-Knight did you manage to solve this issue?

Mirror-Knight commented 3 years ago

Not yet. I'm suspecting the issue to be with the server I'm working on, considering its working on your end but not my end. I fully reinstalled anaconda, tangos and pynbody and still had the same issues. I'll try installing tangos and running things on my local linux machine some time over the next few days to see if that works

Mirror-Knight commented 3 years ago

I believe I've found a a makeshift solution. If I navigate to ~/anaconda3/lib/python3.8/site-packages/tangos/properties/pynbody, I can drop my property.py file directly into there and then add the file name to init.py. Then the tangos write command works and then my properties do show up when I run tangos list-possible-properties.

This seems like manually doing some process meant to be automated by tangos when I set the environment variable and such.

apontzen commented 3 years ago

Great - but that definitely shouldn't be necessary. It would be good if you find out where things are going wrong, but without direct access to your system I don't have any real suggestions for how to proceed. Hard to see how this inconsistent behaviour could arise!

bwkeller commented 1 year ago

I'm running in to this issue as well. I'm running python 3.11.2 and commit 3067d93 for tangos. I did exactly what you showed in your February 2nd comment, and I do not see the veldisp as an available property. I'm going to look into debugging this a bit to try and get to the bottom of it.

bwkeller commented 1 year ago

Alright, I've found the source of the issue and a workaround. The issue is that in properties/__init__.py:502, the current working directory is not in sys.path. If I do export PYTHONPATH=. before running tangos, everything works. I'm not sure if the best behaviour would be to modify sys.path in tangos, or just set the PYTHONPATH environment variable to include the current working directory.

apontzen commented 1 year ago

Thanks Ben! But doesn't python always pick up modules in the current working directory? (Source: https://docs.python.org/3/tutorial/modules.html#the-module-search-path) This is how it seems to behave for me. I'm confused...

bwkeller commented 1 year ago

I don't think it does prepend the current working default to sys.path by default, unless you are running in the REPL or invoking a module: https://docs.python.org/3/library/sys.html#sys.path

It looks like keeping the current working directory in sys.path is considered unsafe, but it might be fine for our purposes.

If I run this in the REPL it prints True:

import sys
print('' in sys.path)

But if I write those lines to a script (testpath.py) and then invoke python3 testpath.py, it prints False