Why is DataTable/View/Set absent?

[FransBouma]

I looked here: https://github.com/dotnet/corefx-progress and saw DataTable/View/Set isn't ported to .NET Core. IMHO this is a mistake, as DataTable and friends are a convenient way to handle untyped data at runtime, and is e.g. often used when fetching data through stored procedures as it's easy to e.g. fetch a dataset with the result of a proc which returns 3 cursors on e.g. Oracle.

I fail to see why this cornerstone of many people's data-access code is absent in a framework that targets servers.

[ellismg]

@terrajobst When you have some free time perhaps you could give an overview about how we came up with what we think the surface area for .NET Core v1.0 is?

[FransBouma]

Whatever may be the reason, having no real story for how to fetch tabular data using stored procs, which is today mostly done using DbDataAdapters storing data in DataSet/Tables, is likely going to stop a lot of people migrating their code to .NET Core for the simple reason that a lot of people have to work with stored procs and the tabular data they return every day. While it might be that a datareader opened on a proc call's resultset and materializing the rows into objects is 'easy', fact is that on databases which return cursors, this isn't that easy, while DbDataAdapters solve that problem today with a few lines of code.

I do understand DataTable/View/Set and DbDataAdapter come with a fair set of interfaces you likely don't want to port now (not sure if they are, haven't checked), e.g. IBindingList, IListSource and the like, but IMHO it's a cornerstone for Data access on .NET, despite what some hip people think.

[tthiery]

I want to extend the question: I understand that .Net Core is not the full .Net Framework and some features are cut out. Nevertheless, many of us rely on some features and want to migrate to .Net Core. But there are two groups: The impossible ones (e.g. WebForms / WPF) and the possible and even reasonable packages like e.g. DataTable/View, XmlSchema, SerialPort, System.Activities, etc. Some may have technical constraints, not be hip, or legally restrained but to know the status and strategy of them would be a great help for understanding this new .Net.

For these cut-out features will there be

• separate NuGet packages outside (or in a later version) of the .Net Core?
• and if yes, will the be created/maintained by Microsoft or should we expect that we need to go to the reference source and do it on our own? (not that this is wrong ... it is just something many employers will not pay for).
• is there an overview of these gaps? I was surprised by XmlSchema (imagine the thousands of existing services), SerialPort (imagine all the inexpensive USB chips which create a virtual COM port) and now by DataTable. The analyzer only helps me with existing code but does not help me in first place when I have to decide on paper which framework to use.

I have not heard of a general strategy for these gaps, maybe I lost it somewhere.

[RickStrahl]

I ran into a use case today where I needed DataTable in order to return dynamic data to a service client. JSON.NET can easily serialize data tables for example. There are no easy solutions for materializing DYNAMIC data in .NET and DataTables make it easy when you don't know what the heck you're going to get back and you need to iterate over it. It's very useful for a number of things...

I can see why DataSet/DataTable is falling out of favor. Personally I rarely use them, but for dynamic data scenarios they are often the best choice available in .NET. So I hope we get this at some point or else an alternative that makes it easier to get dynamic data that can actually be serialized.

[FransBouma]

There's another problem: DbConnection.GetSchema(). It's a way to obtain meta-data from the connected database and all ADO.NET providers have support for it, often with detailed schemas about metadata which can be used by data-access code to e.g. generate code on the fly for types or parameters. A lot of work has been put into this method by ADO.NET providers to make this method work on the database the provider is for: often they use detailed metadata queries so the user of the method doesn't have to.

GetSchema() returns a DataTable, as it's unknown what the types are of the rows returned as that depends on the schema requested and the ado.net provider used.

This method can't be used in .NET core if DataTable isn't there.

[terrajobst]

Why is DataTable/View/Set absent?

We generally consider DataSet and friends (DataTable, DataView, TableAdapter) legacy. However, that doesn't necessarily mean that those APIs will never be available on .NET Core as we generally do support legacy APIs if they important for porting scenarios (for example, non-generic collections are also available for .NET Core).

So why are these types missing then? Simply because we're focusing on non-legacy components first and so far DataTable hasn't popped up yet.

[tthiery]

@terrajobst Answered also my question. A list of "legacy" .Net Framework technologies would be awesome (maybe including a state like "never", "maybe", "later"). I need to understand, in which technologies I can trust.

[FransBouma]

@terrajobst and what about the scenarios I brought up? You can state something is 'legacy' in your eyes but that doesn't necessarily mean use cases a lot of devs are faced with every day are simply 'gone' or have valuable alternatives (as there are no alternatives at this point).

Sure, someone can port the code from referencesource and that might look OK, but it's not, as those types aren't the same types exposed and used in the ADO.NET providers.

I don't have the feeling Microsoft really understands the seriousness of this problem.

[kazuk]

I wish an alternative to SqlDbReader GetScema before makes DataTable and DataSet etc to be legacy and discontinued

My point DataTable is needed for metadata access and some of development work in now can not be obsolete and needs maintain works fine

[robertmclaws]

@FransBouma It is a serious problem for you, because you maintain database products that depend on DataTable functionality. I bet the number of people like you vs the rest of .NET developers is like a fraction of 1%. And a small one, at that.

When rewriting a framework, some stuff gets left behind, especially anything in the edge cases. All of the cases you made are edge cases. You may spend most of your life in them, and many others might because they use your products. But they are not Microsoft's focus moving forward. DataTables are a 15 year old technology, and there has to be a better way to do some of the things you described.

There are reasons EF7 is only usable for simple next-gen ASP.NET apps, and some of the scenarios you mentioned are among those reasons.

For anyone building things that are beyond simple apps, ASP.NET v6 on .NET 4.6 is going to be the way to go. DotNet Core will not be usable for many people in v1, and that is OK. Not every single feature can come along for the ride, otherwise if everything is legacy than nothing is.

Frans, You can't tell me you couldn't come up with a better way to write DbConnection.GetSchema() that could work in your own apps for .NET Core if you really thought about it. Write it as an extension method on DbConnection and make it work with older versions of the framework by using your own objects. Can't be that hard.

[FransBouma]

@advancedrei

It is a serious problem for you, because you maintain database products that depend on DataTable functionality.

My tools don't depend on datatable in their core, they use datatables for meta-data retrieval in some situations (as metadata is only exposed through datatables in these situations)

I bet the number of people like you vs the rest of .NET developers is like a fraction of 1%. And a small one, at that.

Please don't make up numbers and pass them on as real. Almost every application calling stored procedures uses them, and of those applications there are many.

DataTables are a 15 year old technology, and there has to be a better way to do some of the things you described.

There's always a better way, sadly there's not one available at the moment.

There are reasons EF7 is only usable for simple next-gen ASP.NET apps, and some of the scenarios you mentioned are among those reasons.

I think you're mistaken. Plus it's besides the point.

You can't tell me you couldn't come up with a better way to write DbConnection.GetSchema() that could work in your own apps for .NET Core if you really thought about it. Write it as an extension method on DbConnection and make it work with older versions of the framework by using your own objects. Can't be that hard.

Sometimes GetSchema() is necessary, but it's just an example. You also have GetSchemaTable on a datareader, which is used in a lot more cases (as not everyone reads metadata from a schema), as it tells you what the layout of the resultset is. Essential information for creating datastructures to store resultsets of unknown layout. And yes, that's not in less than 1% of the applications out there. Try most enterprise applications which have to talk to databases with many stored procedures for example. Good luck with writing solid server side code without datatable.

Sure, like I said there's always a way, but the thing is: people will want to port code they already have to .NET core and e.g. run it on Linux. Now they can't do that.

[nvivo]

@terrajobst,

Out of curiosity, is there any public information you are using to decide what is important to port or not? Mono used to use Moma to scan and publish a database of "what people really need" to help drive the implementation.

Is there any metric like that being used to help decisions, or is it based more on the direction the team want things to go?

[tthiery]

@FransBouma +1 for the answer. The 1% is totally unrealistic considering that EF showed up in 2011 (2/3 of .Net lifetime are before) and the consideration that DataTable was the dominant technology for data retrieval in all earlier examples. I would more go for 20% to 30% of all enterprise platforms will have dependencies to DataTable (numbers are random as well). Nevertheless, @advancedrei is not completely wrong. He is very right that it is a wrong assumption that the .Net Core will be anytime close a reasonable migration or extension path for existing code. Primary focus are and will be for a while ASP.Net (where the EF is quite dominant in new designs) and .Net Native (which do not rely on local databases but data centers using services) as far as I understand.

@nvivo +1. I totally agree. It is a bit ... in-transparent currently (especially for a Open Source project which expects contributions). @terrajobst Like stated above ... having a forecast what might (DataTable, XmlSchema, etc) and what definitely will be never ever be in (remember the endless .. WPF/WinForms/WebForms discussions) would really be helpful.

[robertmclaws]

Entity Framework came out of Object Spaces which puts it in the 2008-2009 timeframe. And the 1% I referred to was forward-looking development. How many of those 30% Enterprise projects aren't even on .NET 4.0 yet? I'd venture to say a lot.

NETCORE is not concerned with people who will never migrate anyways. It is concerned with building the next generation of apps. Not maintaining the last one. The Frameworks that have already shipped can handle that just fine.

[FransBouma]

@advancedrei .NET core on Linux could spark new interest in .NET from dev teams who now won't look at .NET as it's windows only but do have lots of databases with stored procs. There are a tremendous amount of database-driven applications out there which have to use stored procs (I use stored procs as an example, as it's one of the use cases for a datatable as the resultsets are often of unknown layout: as you can't determine the layout .... without a datatable ;)) and they can't pick .NET if there's not a solid scenario available for their situation.

There's little known about the ADO.NET story for .NET core, no code in that area is opened yet (to my knowledge) and how the provider model will look like is completely unknown. For server-side software that's an important point, and it now looks like v1.0 of .NET core is completely useless which IMHO wastes a tremendous opportunity to win people back to .NET.

But alas... I'm starting to repeat myself. :)

[nvivo]

@tthiery, @terrajobst

Is there any metric like that being used to help decisions?

I'm sorry. I know it sounded like I was questioning the decision, but I was not. =) I was genuinely curious how the process of selecting the API was/is done.

I have been using pure ADO.NET for years without EF, and I have not used DataTable for anything new in ages. I think this can be provided later by a package, but don't need to be in the core right now. We need to remember .NET 4.6 will still be released and all this stuff will be there.

[RickStrahl]

I'm with others here who point out that we should be looking forward not backward, but... I think we need to be careful here that we don't kill functionality that may be crucial to building applications.

The two things that have jumped out at me are lack of a good story for creating new providers so that third parties have an easy time to create new providers. This (or some other mechanism like standardized DI) is a MUST and currently not supported/documented. Right now you're stuck with SQL Server and third parties can build their own providers but have no effective way to swap providers at runtime (maybe there's something there that we can't see but this was what the provider model was for previously).

Personally I don't see DataTable/DataSet as needed tech, but the features it provides - an easy way to retrieve list based dynamic query DB data is a solid use case for building solutions on top of raw ADO.NET and for third parties that build on top of it. I think something is needed to provide this functionality out of box. It could be as simple as a dictionary, but it should be there natively IMHO because it is a common scenario if you use ADO.NET raw.

So to me the story isn't about backwards compatibility but making sure the use cases that are common are addressed. If that means new interfaces that's fine, as long as some fundamental scenarios that third party tools and frameworks can build on are addressed in some way.

Honestly I think this is all of utmost importance. After all most applications use data. And the story for data in vNext as presented by Microsoft so far looks like a bungling mess. I'm not sure if it actually is or not, but the message has been muddled and confused. From the outside it looks like EF with SQL Server is the only viable solution that's in box and that really sucks if that's how it ends up. Especially in light of .NET Core running on other platforms where SQL Server usage is not very likely. If this is just a messaging thing - then that needs to be corrected with some official commentary. If it's a feature thing then I think there should be discussion on what needs to be there.

[nvivo]

I agree with @RickStrahl that EF + SqlServer is bad, and agree with @FransBouma that there is no way to do some things like just selecting untyped data without dealing with DataReader. But I don't see anything else filling this gap DataTables left.

What is really to be missed? All the select + order by is done much better by linq directly on objects. For other things, there are alternatives to EF. SimpleData is a famous one, Dapper from StackExchange is also very handy, and even I maintain a pretty useful one =).

I bet new alternatives will appear even more with .NET Core, and eventually a winner will come up. But we need to give it some time.

[FransBouma]

I agree with @RickStrahl here. It's the concept / featureset provided by Datatable and friends that's essential, and if I may add, essential for v1.0 to be there. Datatable is an obvious choice to provide that featureset as a truckload of code is already written out there supporting it, but it's not essential to have it as the type to provide that featureset, however why write something else to replace something you already have?

[nvivo]

@FransBouma,

Which part exactly do you miss from DataTable? It seems the only thing that you miss is a type that represents "an array of object arrays" for untyped data, but that is provided out of the box.

Is that it or am I missing something?

[nvivo]

As a note, I saw the comments about the GetSchema. The point is that those things can be provided as separate modules, they are not exactly required by anything other than supporting the IDE around DataTable/Set/View stuff. And even this could be today exposed as real objects instead of datatables.

If all is needed is a way to retrieve untyped tabular data, we could strip away 90% of DataTable and have a single new type that have no change detection, no select capabilities, etc. It should only hold an array of "rows", so "an array of object arrays".

[robertmclaws]

I agree with @nvivo, a DataTable is just an array of object arrays. Not sure why the functionality can't be replicated with some custom code and without the "sky is falling" mentality.

[FransBouma]

@nvivo

Which part exactly do you miss from DataTable? It seems the only thing that you miss is a type that represents "an array of object arrays" for untyped data, but that is provided out of the box. Is that it or am I missing something?

DataTable isn't a construct with arrays of object arrays, it's a construct with typed columns (data isn't stored internally in object arrays, but vertically in typed columns), which has the ability to define custom views on top of it you can sort/filter without touching the original datatable, which can be filled with a DbDataAdapter which can create the typed columns for you during fetch based on metadata obtained from the resultset, which is also stored in a datatable.

Like I said, if something is replacing it with equal functionality, fine, but that's not in the cards. This means that ado.net providers can't provide the same functionality they do today. GetSchema is one thing, but as I said, DbDataReader.GetSchemaTable is another, and that one IS used a lot by user code (directly or indirectly through data-access code or dbdataadapters).

@advancedrei please stay on point with this discussion. I understand you don't give a shit about datatable and friends and that's fine, but just because it's irrelevant to you doesn't mean it's irrelevant to many thousands of developers out there.

[nvivo]

@FransBouma,

DataTable isn't a construct with arrays of object arrays

I'm obviously not saying how DataTable is implemented, I'm saying this is the only part that is missing.

if something is replacing it with equal functionality, fine, but that's not in the cards

Don't get me wrong here. I have a lot of code that depends on GetSchemaTable and DataTables in general out there. I have data migration tools that automatically compare and move data between different providers using this functionality.

But that doesn't matter because we are not discussing removing this functionality from .NET. We are discussing if this functionality should be added to a new framework that has a different target.

You say that you want "equal" functionality, but most of what DataTable/View/Set provides can be achieved with POCO and LINQ in a much cleaner way. You can create views, group by, count, select different rows and columns, etc. And in most cases, that will be achieved with LINQ using less memory.

I believe the only way DataTable should be added to CoreFx would be with a simpler design that integrates better with LINQ and removes things like .Select, OrderBy, Views, etc. It should be just a container for tabular data, nothing more.

As someone that works a lot with DbProviderFactory, I liked how the design went. Providers are clean and do only what they need to allow database access using SQL. Everything else can be build on top of it later, and that's how it should be.

Version 1.0 is not even out yet, and @advancedrei said already, this version won't have a lot of stuff for a lot of people. But it's better to start clean and add pieces as it goes than start bloated.

[FransBouma]

@nvivo

You say that you want "equal" functionality, but most of what DataTable/View/Set provides can be achieved with POCO and LINQ in a much cleaner way. You can create views, group by, count, select different rows and columns, etc. And in most cases, that will be achieved with LINQ using less memory.

You can't create POCOs if you don't have meta-data of the fields of the resultset. You can't determine the schema of a resultset of a stored procedure unless GetSchemaTable() is implemented. Again, if that's implemented but it gives me a typed object, no problem. However if it returns an array of object arrays, what am I going to do with that? What element is what and of what type?

I'm not talking about the POCO/typed element route which is created from metadata obtained in some way, I'm talking about calling a stored procedure which returns a resultset and you now have to do something with that resultset.

Anyway I have given enough examples, @RickStrahl has given additional examples and background of why this feature is very important. I simply don't get why one would want to be limited by excluding functionality which is already implemented and has worked for over a decade.

[nvivo]

@FransBouma,

Everything you said is perfectly achievable if you change your mind model.

You can't determine the schema of a resultset of a stored procedure unless GetSchemaTable() is implemented.

Why is that? Any database nowadays implements INFORMATION_SCHEMA to provide this kind of information in a standard way. If you need something very specific, you can just run any query against the database.

I'm not talking about the POCO/typed element route which is created from metadata obtained in some way

So, you are saying that if you ignore all the other ways to do it, there is no way to do it?

DataTable is a huge implementation that is intended to do a lot of things that are not that common anymore. I have been building commercial apps for ages with plain ADO.NET and didn't use DataTables for a long time, I see them as legacy as well. Most frameworks and controls today focus on objects with observables and other ways to track changes.

I do believe it has some features that are still valuable and have no replacement, but we need find what those features are and try to come up with an alternative for those only. Trying to bring this huge implementation full of duplicated stuff that is easily achievable with LINQ is not a good option.

[FransBouma]

Everything you said is perfectly achievable if you change your mind model.

It's not for me, it's for all those devs out there who can't proceed to .NET core because someone thought 'Datatable' is legacy while not considering the consequences of that decision.

You can't determine the schema of a resultset of a stored procedure unless GetSchemaTable() is implemented.

Why is that? Any database nowadays implements INFORMATION_SCHEMA to provide this kind of information in a standard way. If you need something very specific, you can just run any query against the database.

1) Not every database implements all schemas of information_schema 2) Not every database is 100% bugfree in those schemas (e.g. sql server) 3) For a resultset from a stored procedure, e.g. the direct resultset like with sql server or the resultset of a cursor like with Oracle or PostgreSql, the schema isn't in one of those schemas, but is determined on the fly, as the resultset is dynamic. If you don't know what you're talking about, please don't derail this discussion with nonsense as it's important for people who have to work with DataTable and friends. 4) I've written meta-data retrieval drivers for a lot of databases and for a lot of different meta-data elements. Trust me on this that INFORMATION_SCHEMA is not that helpful, if helpful at all.

The DbConnection.GetSchema() call is an example, not the use case. There are many more examples to give where DataTables are used today and are very useful. I have given a couple, others have given other examples.

DataTable is a huge implementation that is intended to do a lot of things that are not that common anymore. I have been building commercial apps for ages with plain ADO.NET and didn't use DataTables for a long time, I see them as legacy as well. Most frameworks and controls today focus on objects with observables and other ways to track changes.

Common in your world != common in everyone's world. I write ORMs for a living, I know DataTables are not the only way to do data-access. I also know that everything that even looks like a stored procedure is automatically useless if datatables are not in the framework anymore.

I do believe it has some features that are still valuable and have no replacement, but we need find what those features are and try to come up with an alternative for those only.

Why come up with alternatives for code that has already been proven to work for over a decade and which is used by a lot of people, which is supported today in ADO.NET providers (so when these are ported to .NET core, this code will work automatically).

Trying to bring this huge implementation full of duplicated stuff that is easily achievable with LINQ is not a good option.

You're not focusing on the real problem. I'm not talking about creating a projection of a resultset to a typed object, I'm talking about being able to store untyped resultsets in a datastructure which makes it possible to use that untyped resultset without knowing what the layout is (as e.g. that might differ based in input, some stored procedures are written that way).

Anyway, I've done my best. If MS wants to shoot themselves in the foot by limiting the Data-access story of their .NET core framework to 'whatever EF7 can do', so be it.

[HaloFour]

After reading through this thread I tried to find out what of ADO.NET is being brought over to CoreFX (or at least would be supported on top of it) and I couldn't really find anything. Are connections, commands and readers also getting the axe (or at least deferred until after the first release)? Doesn't EF7 depend on these things to function?

A server core framework for web applications that doesn't support DB access doesn't sound very useful to me.

I can't honestly say that I've used DataTables and their kin recently but I do use data readers frequently when I need both firehose performance and/or need to avoid the inherent limitations of ORMs. I also have a metric ton of extension methods around DbConnection, DbCommand and DbDataReader to vastly simplify their use which I'd love to propose for potential inclusion/discussion.

Update: My apologies, apparently I can't read. I do see the progress page which includes the types in System.Data.Common.

[nvivo]

@HaloFour, check for System.Data.Common under https://github.com/dotnet/corefx-progress/tree/master/src-diff/README.md.

Specifically, check the definition of DbProviderFactory that defines what a Data Provider is.

Basically it was reduced to connection, command, transaction and datareader.

It seems all that is needed to me.

[nvivo]

@FransBouma,

It's not for me, it's for all those devs out there who can't proceed to .NET core

Who are you? Superman? Let's be honest here, any issue we open is because some personal need. But please bring some of those oppressed developers to the discussion, so they can participate, maybe having more points of view on this would help.

Not every database implements all schemas of information_schema

Most schemas provided by .NET are just queries to these tables. I have also worked a lot with those, and there are huge differences between different providers to the point that unless you just want a list of tables, you have to write different code to get what you want anyways. Why should .NET add another level of indirection? If the database can't provide this information in a sane way, what guarantees you have that the driver will?

You may argue this is useful in some cases, but is not required to have a data provider, and it's not required to use procedures in any way, even though you insist it's impossible to execute a store procedure without it, which I honestly don't get it as I have used them without that since forever.

You're not focusing on the real problem. I'm not talking about creating a projection of a resultset to a typed object, I'm talking about being able to store untyped resultsets in a datastructure which makes it possible to use that untyped resultset without knowing what the layout

Back to my point, we need to understand exactly which part of the DataTable stuff is missing and address this specifically. The things you are talking about don't need to include change tracking, select, orderby, table relationships, views, etc. This should be addressable with a simple container for tabular data, and it shouldn't be that hard to build that and even include in .NET later.

[FransBouma]

@nvivo

It's not for me, it's for all those devs out there who can't proceed to .NET core

Who are you? Superman? Let's be honest here, any issue we open is because some personal need. But please bring some of those oppressed developers to the discussion, so they can participate, maybe having more points of view on this would help.

What does this add to answering my question, which wasn't directed to you at all btw. Is it necessary here to explain if I know what I'm talking about? It seems so. As if I don't know the field I'm specialized in and spend full time in.

Not every database implements all schemas of information_schema

Most schemas provided by .NET are just queries to these tables. I have also worked a lot with those, and there are huge differences between different providers to the point that unless you just want a list of tables, you have to write different code to get what you want anyways. Why should .NET add another level of indirection? If the database can't provide this information in a sane way, what guarantees you have that the driver will?

connection.GetSchema() gives also schemas about the type mappings provided by the ado.net provider, and an easy to use interface to otherwise complicated queries. But it's an example of where datatable's are used. There are many more.

You may argue this is useful in some cases, but is not required to have a data provider, and it's not required to use procedures in any way, even though you insist it's impossible to execute a store procedure without it, which I honestly don't get it as I have used them without that since forever.

sigh. of course you can execute a stored procedure, consuming the resultset returned by it in any reasonable way isn't possible, as the only interface you'll get in .NET core is DbDataReader.GetValues(). Good luck with that. Before you say 'but I know the schema', you don't: you can't determine the resultset from the proc. You might be able to determine the field names but no types. If a field in the object array returned from GetValues() is DBNull.Value, you can't determine the type of it. No metadata schema in the DB will help you there.

But of course, you did that many times over without problems.

Back to my point, we need to understand exactly which part of the DataTable stuff is missing and address this specifically.

'we'? Who are 'we' here? I don't see how you should understand anything. I asked a question to Microsoft and I got the dreaded 'it's legacy' excuse I've heard too many times before already. Other people have also asked questions in this thread without any answer at all.

I've described which parts are missing and others have too. That you don't get that, that's fine but irrelevant: you weren't the person I addressed in my initial question.

The things you are talking about don't need to include change tracking, select, orderby, table relationships, views, etc. This should be addressable with a simple container for tabular data, and it shouldn't be that hard to build that and even include in .NET later.

Yes, let's rewrite something from scratch to avoid porting perfectly fine code over we already have laying around. That's time well spent! :/ Especially because your 'simple container for tabular data' will very likely look like a datatable, which is already highly optimized. But again, whatever is added is fine as long as it addresses what's missing, and as there are millions of lines of code out there already working with datatable which can't be ported to .NET core without it (but of course you won't see the need of that at all, as you have done things differently many times over) I don't see why on earth something else has to be written from scratch while there's already a class which does this.

If this is the future of Microsoft Open Source, where asking a simple important question results in a serious lack of answers from MS coupled to defending the question to people who come in and think they have to show their lack of knowledge, it'll be a fun place to be, for sure.

[nvivo]

'we'? Who are 'we' here? I don't see how you should understand anything. I asked a question to Microsoft

This is an open source channel to discuss features of an open source project with interested people. If you wanna talk to Microsoft only, you should call their support.

[FransBouma]

To add: it's already frustratingly difficult enough to convince MS to include something or to change something. It's even harder if people who have no stake in the matter come in and try to defend what MS has apparently decided. If this gets included, nothing will change for you. You won't lose any time, code, hair whatever. Nothing. However people who do rely on this feature are helped. Yes, I asked this question on their behalf because I spotted the absence of DataTable, and most people likely won't look at .NET core till it's v1.0 Adding it then is much harder than when v1.0 hasn't been released.

I must say this whole debate has been very frustrating for me. I asked a serious question and got 1) a standard excuse from MS without any reasoning behind it and 2) have to defend why I even dared to ask the question to people without any stake in the matter.

From the lack of response from the .NET team I conclude that unless IBM or other big corp calls MS and asks them wtf they were thinking by not including datatable, we won't see it in v1. This is a shame, but at the end of the day, it's MS' responsibility to provide value for everyone.

This is an open source channel to discuss features of an open source project with interested people. If you wanna talk to Microsoft only, you should call their support.

Yes, but you're defending their decision while you have no stake in the matter: if you are or aren't convinced datatable should be added is irrelevant. All your posts do is make this whole debate a matter of whether you are convinced or not, but it should be whether Microsoft is convinced or not. This derails the discussion as it centers around what you think, which is irrelevant: you don't decide what's included in .NET core, MS does.

So again, unless you work for MS and have a stake in the matter, please stay out of the debate; that you aren't convinced it's useful because you don't use it is not an argument. I don't write websites, it doesn't make ASPNET not needed.

[nvivo]

Oh, for crying out loud! Stop complaining already. You didn't provide a single point on why any of these things can't be done in any other way. All I see is that you are losing your favorite tool and you don't want to learn anything new.

Again, this is not a microsoft support channel. This is an open source project and anyone can discuss here anything related to the future of the platform. Get over it.

[FransBouma]

Oh, for crying out loud! Stop complaining already. You didn't provide a single point on why any of these things can't be done in any other way. All I see is that you are losing your favorite tool and you don't want to learn anything new.

Sorry? My framework doesn't even rely on datatable. (only in the pipeline which projects to datatable, which is an additional feature). It's an O/R mapper, remember?

And I don't want to learn anything new? Where is that the case? I'm a software dev for over 21 years now. If I don't want to learn anything new I'd still be writing C on a Sun workstation. But thanks again for posting an offtopic post.

Again, this is not a microsoft support channel. This is an open source project and anyone can discuss here anything related to the future of the platform. Get over it.

I don't think you grasp the concept of why I asked this question. Here's a hint: it's not for my own code, I don't write LoB apps, it's not for you, it's not for the other 'I don't use it so it's garbage' intellectuals here, it's for the people who don't read this thread and don't know DataTable isn't supported and will be faced with a problem when v1.0 is released. That is of course irrelevant to you as you couldn't care less about these people. Duly noted.

[Tragetaschen]

I'm a bit confused: .NET Core is about factoring the .NET Framework into individual, small and focused Nuget packages. So if anything, there will be such a separate package for DataTable and friends. A v1 Release of .NET core means the underlying bits (corlib, JIT, etc.) are done and some Nuget packages are released as well. These packages are currently driven by the ASP.NET 5 efforts.

I don't get why it would be a problem when (as well as many other parts of the framework) DataTable wouldn't be available yet when v1 is pushed out. This is something that can be easily added on a completely independent time base.

[nvivo]

@Tragetaschen,

The issue is that removing DataTable causes some other parts of the framework to be removed, like GetSchemaTable and a type that can hold untyped data offline with no need to deal with datareader.

Those things can be added later as you said, and may impact some people but there are alternatives today.

As @FransBouma repeatedly said, he doesn't use these features neither it affects his projects. But he is worried that some other people will have their lives devastated by not having this specific support in v 1.0 for some reason.

I think it would be much more useful to discuss things that really affect someone so we can discuss alternatives and even create new proposals to add those things back in a saner way, instead of trying to bring the entire DataTable bundle to corefx.

[Tragetaschen]

Still, this thread now focuses way too much on a "removal" while it's actually about "not yet ported over". This is what @terrajobst made perfectly clear in his response.

Of course, a package for DataTable will either have a dependency on a package including GetSchemaTable, or (if it's only required in that one scenario) will include and provide this and other types.

[FransBouma]

A package can't be supplied later without also altering e.g. DbDataReader's interface, as it is currently exposed there. That is, if the package wants to supply the functionality that's there today. (Another affected element is DbProviderFactory, which currently supplies DbDataAdapter, a class which is also not ported over but which is often used in combination with DataTable/Set, e.g. when fetching resultsets from procs with e.g. cursors.)

If it's just a standalone class, sure, it can be added later. However adding this later will break the interfaces of several classes available at v1, meaning there's a versioning problem on the interface if this is added later on, which is avoidable if the type is included in v1 (and which also avoids a lot of people having to wait for the addition of the type to move to .net core).

[nvivo]

@FransBouma,

Why this needs to break the DbDataReader interface just to get the schema? You insist on that, but I don't see why this is the case.

Your assumption is that the only way to do this is by returning a datatable from reader.GetSchemaTable, but there are other ways to do that.

I'd support a way to do that with extension methods based on some interface implementation that is optional and returns some new type other than DataTable. Most types in .NET are following this convention, and it works.

[FransBouma]

Extension methods don't work here as the method has to be implemented by the ADO.NET provider being used, which means it has to be implemented by the datareader, hence the design of the code as it is today. An extension method can be used of course, but code using that extension method then is tied to the ado.net provider, throwing dbproviderfactory out of the window.

So yes, it will break the interface.

But for the gazillionth time, it was an example, not the use case. E.g. it totally doesn't address Rick's concerns at all or how to fetch a dynamic resultset from a query so you can at least do something with the data other than 'oh, field X has value NULL, no idea what type'.

[nvivo]

Extension methods don't work here as the method has to be implemented by the ADO.NET provider being used, which means it has to be implemented by the datareader.

Why? I don't see any reason for that to be true. There are so many ways to implement it that doesn't require a single interface to be exposed.

But for the gazillionth time, it was an example, not the use case

Yes, you didn't provide a single use case yet. I get that. It would be more productive if you did.

[FransBouma]

Why? I don't see any reason for that to be true. There are so many ways to implement it that doesn't require a single interface to be exposed.

What will implement the extension method? The only way it can report on the layout of the resultset if it can look into the raw data of the resultset inside the datareader. Anything outside the datareader and the ado.net provider thus can't do that.

Yes, you didn't provide a single use case yet. I get that. It would be more productive if you did.

wat... :-1:

(edit) Anyway, have fun. I'm out of here. MS apparently doesn't give a shit, and I have no time for debating this issue with people who just want to troll along.

[nvivo]

You don't need to expose that with the DbDataReader interface. That's my point.

This an additional feature, it can be exposed with an additional interface.

[ilmax]

Disclaimer : I don't build ORM, but I do use them on a daily basis.

I think that DataTable & friends are an obsolete technology, but the lack of alternatives to dynamically extract data from a DbDataReader feels to me like ADO.NET is losing power.

To be clear, I understand @FransBouma request, but I also understand @nvivo motivation. As @FransBouma pointed out, (at lest IMHO) is not the absence of DataTable the point, but the absence of any valid alternatives, and given that not in every data retrieval scenario the data shape is known in advance, sometimes POCO are simply not enough/convenient.

As adding functionality via extension methods, this could be achieved of course, but since every DbDataReader implementation should have it's own say GetSchema extension method, you should cast the DbDataReader to it's underlying type to call the correct method, and this makes working with DbProviderFactory and different ADO.NET implementation at the same time a little bit more difficult.

Note either that @terrajobst response

So why are these types missing then? Simply because we're focusing on non-legacy components first and so far DataTable hasn't popped up yet. does not states that DataTable & friends will never be part of netfx.

[william-gross]

@FransBouma, thank you for your posts here. I agree with you.

I maintain an open-source framework (https://www.nuget.org/packages/Ewl/) that uses GetSchemaTable (for SQL Server, MySQL, and Oracle) to determine the shape of query results so it can generate code. There are about twenty web applications built on top of this framework, and none of them will be able to move to .NET core if it does not include GetSchemaTable or something else with the same functionality.

[nvivo]

There is no doubt some people will miss this feature. But for every developer that will miss GetSchemaTable, there are 50 missing WinForms or WPF or something else. I'm missing some features that are not ported as well, and I'll need to find alternatives. But if .NET Core is driven by that, the entire framework must be brought together.

I believe the team didn't remove a feature just for the sake of removing it. There is a reason behind this, and we should either ask them if there is anything planned to provide this support or come up with an alternative design that don't require all the legacy stuff implemented in a new framework.

[FransBouma]

There is no doubt some people will miss this feature. But for every developer that will miss GetSchemaTable, there are 50 missing WinForms or WPF or something else. I'm missing some features that are not ported as well, and I'll need to find alternatives. But if .NET Core is driven by that, the entire framework must be brought together.

Frankly I don't care. I have functionality to support and if there's no alternative because MS doesn't want to port it, I have a serious problem with porting my system to .NET Core. I understand things are pretty small at the start of .NET core, and things will be increasing over time. However with this particular issue it doesn't look like it will be making it to .NET core _at all_, ever. And that's the problem I have with it. 

I believe the team didn't remove a feature just for the sake of removing it. There is a reason behind this, and we should either ask them if there is anything planned to provide this support or come up with an alternative design that don't require all the legacy stuff implemented in a new framework.

I think I did that ;) (ask what is going to happen). So far no reason has been brought forward other than the usual 'not our problem' bullshit. Microsoft should take a note from Oracle which goes beyond everything to keep Java backwards compatible so stuff keeps working. 

It's of course that by supporting datatable a lot of other stuff has to be brought over as well. That's a lot of work, but hey, they started this mess. 

I don't mind if it's not in .NET core v1.0 as it's very limited anyway. I do mind if because Microsoft thinks it's legacy they can ignore features completely and shrug when the users of these features suddenly have a big problem. 

If I have to port the stuff myself, so be it. The problem however is that GetSchemaTable is in the ADO.NET interfaces so if I port the .NET reference source over, I still lack the essential method in the interface. So MS holds a key where no-one can get access to and in the future no-one will be able to add this feature themselves to the code as well. 

So in short: this whole mess sucks big time. The way it is handled, the way the silence bugs on and the way there's no clear vision how this is going to be solved in the future (if ever). 

Yeah, cheerio! .NET core! let's all be happy.... :/

    FB

[nvivo]

@terrajobst I agree on the silence issue.

This is not about porting DataTable/DataView, it's about how do we access metadata about queries in this new api?.

It would be nice to hear anything about your plans on this.

[shanselman]

Ping