jehugaleahsa / FlatFiles

Reads and writes CSV, fixed-length and other flat file formats with a focus on schema definition, configuration and speed.
The Unlicense
357 stars 64 forks source link

SeparatedValueTypeMapper defined class has to be public? #25

Closed ithielnor closed 6 years ago

ithielnor commented 6 years ago

We upgraded from 0.3.23 to 1.6.2 and started getting an error on our private dto classes.

Here's how we're using it

public class MyClass
{
    class Status
    {
        public string Identifier { get; set; }
        public string Status { get; set; }
        public DateTime? EffectiveDate { get; set; }
        public DateTime? ModificationDate { get; set; }
    }
    public void DoSomething()
    {
        var mapper = SeparatedValueTypeMapper.Define<Status>();

        mapper.Property(x => x.Identifier);
        mapper.Property(x => x.Status);
        mapper.Property(x => x.EffectiveDate).InputFormat("yyyyMMdd");
        mapper.Property(x => x.ModificationDate).InputFormat("yyyyMMddHHmmss");

        ...

        var data = mapper.Read(reader, new SeparatedValueOptions()
        {
            IsFirstRecordSchema = false,
            RecordSeparator = "\n",
            Separator = ",",
            Quote = '"'
        });

        ... enumerate it
    }
}

Previously the class MyClass.Status did not have to be public, but now if it's not public, we receive an error like this one:

System.MethodAccessException: Attempt by method 'DynamicClass.()' to access method 'DynamicClass.()' failed. at () at FlatFiles.TypeMapping.Mapper`1.<>c__DisplayClass11_0.b1(Object[] values) at FlatFiles.TypeMapping.TypedReader`1.Read() at FlatFiles.TypeMapping.SeparatedValueTypeMapper`1.d69.MoveNext()

Is this intended behavior? Why does the FixedLengthTypeMapper still work fine with private classes?

jehugaleahsa commented 6 years ago

Using your example, I created a new unit test and pointed it to an internal and a private class but they worked just fine.

I noticed both the class and the property were named Status - I assumed that was just for the example.

Is this class being generated at runtime, perhaps, rather than at compile time?

Also, it might have to do with the class being internal rather than private (the default if not indicated is internal). The .NET runtime has some pretty crazy restrictions on accessing internal members via reflection.

Any additional information you can provide would be appreciated.

ithielnor commented 6 years ago

The names are just for example. Yes, it's unspecified. It's a compiled class.

jehugaleahsa commented 6 years ago

Can you tell me a little bit more about your environment? Is this for an ASP.NET application, Silverlight, .NET Core, etc.? I'm just trying to figure out what the magic ingredient is; I'm unable to create a unit test that fails.

I actually find this surprising because in a recent version I added code to support more private and internal properties/fields/constructors. Both the fixed-length and separated value mappers are using the same underlying classes to create/read/write objects, so it's weird one would work and the other wouldn't.

ithielnor commented 6 years ago

We're targeting .Net Framework 4.6.2. This is run from a windows service.

I'm gonna try a few more things on my end and see if I can find out what the cause is myself. From your tests, it sounds like my initial assumption was incorrect.

ithielnor commented 6 years ago

I setup a test with a similar file locally and can't hit the problem here either.

It must have something to do with my stream reader. Our files are stored on S3 and buffered into a memory stream for parsing. Something like this:

using (var reader = new StreamReader(bll.GetStream(file)))
{
    IEnumerable<Status> data = mapper.Read(reader, new SeparatedValueOptions()
    {
        IsFirstRecordSchema = false,
        RecordSeparator = "\n",
        Separator = ",",
        Quote = '"'
    }).ToArray(); // Errors upon enumeration here
}

public Stream GetStream(string amazonFileName)
{
    MemoryStream responseStream = new MemoryStream();

    GetObjectRequest request = new GetObjectRequest()
    {
        BucketName = Bucket,
        Key = amazonFileName
    };

    using (GetObjectResponse response = _client.GetObject(request))
    {
        BufferedStream bstream = new BufferedStream(response.ResponseStream);
        byte[] buffer = new byte[0x2000];
        int count = 0;
        while ((count = bstream.Read(buffer, 0, buffer.Length)) > 0)
        {
            responseStream.Write(buffer, 0, count);
        }
        responseStream.Seek(0, SeekOrigin.Begin);
    }

    return responseStream;
}

I don't see why that would cause a problem, but it's the only difference I can see from the working tests.

jehugaleahsa commented 6 years ago

I'm closing this since it's been open for a while and there's been no progress.