UOW for multiple entities instead of Transaction

[AliCharper]

Good day,

First of all, I should appreciate you for the time and resources you have spent to develop such this platform. In the second, may I ask you a question ?.

If we add more Entity to have a real case study, can he trust the UOW and not to implement any kind of Transaction if we want to ADD/UPDATE/DELETE some records from multiple entities or no ?.

Regards Ali

[Boriszn]

Hi Ali,

Thank you a lot for the positive feedback.

Simple answer: No, you don't need any transaction mechanism. UoW basically represents transactions.

Let me explain a bit how it works:

1. When you execute add, updateor delete(methods from Repository) entity framework just adds this changes to tracking log Tracking log, in memory. (You can see this changes in db context, tracking entities etc ...)
2. When you execute Commit() EF will take all changes from TrackingLog (in memory) and executes them as queries to Database. So, only, after this action, the Database will be updated.

3. In case if exception (somewhere in Business Logic/ Services) will be thrown UoW dispose all tracking changes (and DB context itself) So this is basically Rollback mechanism.

   (You can test it, just put throw new Exception('test') somewhere in service, and breakpoints in UoW Commit/dispose methods)

(However, you may need some transactions in case if you need to switch between different databases like MS SQL, Oracle or MySql, in real-time and you using different EF providers)

Please let me know if you have more questions.

Regards, Boris

[AliCharper]

Thanks, but I still have had some issues for Nested Entities, because the record in Parent table should be inserted at first so I need two Commit and this means a big problem for me. I need a mechanism to be able to ADD/Update/Delete in one shoot even for Nested scenarios.

Regards Ali

[AliCharper]

Another thing is : Sometimes, I should make sure about inserting the first record in a Table and then start adding another one in the second table. In such this cases, I am paralyzed or if I need the Identity number which DB generates after inserting the record.

[Boriszn]

As I understood, in first example you are using 1 transaction, right ?

So one transaction in current solution may look like:

// 1.  Starts the transaction for Device and DeviceDetail object

 var deviceRepository = unitOfWork.GetRepository<Device>();
 var deviceDeviceDetailRepository = unitOfWork.GetRepository<DeviceDetail>();

// Get device
Device device = deviceRepository.Get(333);

if (device == null)
{
      // 2. Will rollback transaction if exception 

      throw new NullReferenceException();
}

// updates the device 
 device.DeviceTitle = deviceViewModel.Title;

 deviceRepository.Update(device);

// adds new deviceDetail 
deviceDeviceDetailRepository.Add(
   new DeviceDetail { Description = "New description" } )

 // 3. Commit transaction (all changes in Device and DeviceDetail  will be moved in database)
unitOfWork.Commit();

Here I update Device entity and added new DeviceDetail entity with only one commit. How it may look like in SQL transactions:

BEGIN TRANSACTION;  

BEGIN TRY  

    UPDATE  Device ....
    INSERT INTO DeviceDetails  ....

END TRY  
BEGIN CATCH  

    IF @@TRANCOUNT > 0  
        ROLLBACK TRANSACTION;  

END CATCH;  

IF @@TRANCOUNT > 0  
    COMMIT TRANSACTION;  
GO 

[Boriszn]

In another example you are using 2...n transaction, right ?

Then you should use 2..n commits, and you can even check, if data was updated before first commit, with simple check: if(unitOfWork.Commit() != 0) { // execute another transaction code }

You can also split Commit(dbContext.SaveChanges) with add/update/delete Repository methods, and remove UoW. But I do not recommend to do it. (Perhaps, in projects with legacy code) Also, I've recommend you to read about UnitOfWork pattern itself (like: martinfowler-UoW )

Maybe you can provide/share you code examples, etc ?

Best regards, Boris

[arunprasathv]

@Boriszn It's a great Article. I've got a use case where I need to join multiple tables (5+) to get the final result. How would you do that in this architecture and will it be scalable? Will there be any performance impact in this approach?

[AliCharper]

Thanks alot. Due to Iranian new year, I have been busy from yesterday. I will have a look tomorrow.

[AliCharper]

I think your first example is 100% enough for any kind of scenarios , but let me check it on a real scenario. Will update you all asap.

[Boriszn]

@arunprasathv Thanks a lot. Basically, Yes this is possible. Example:

// 1. Create repository
 var deviceRepository = unitOfWork.GetRepository<Device>();
 var deviceDeviceDetailRepository = unitOfWork.GetRepository<DeviceDetail>();

 // 2. Get all devices and deviceDetails
 IQueryable<Device> devices = deviceRepository.GetAll();
 IQueryable<DeviceDetails> deviceDetails = deviceDetailRepository .GetAll();

 // 3. Join 2 tables, uses device as a result model
var deviceWithDetails =
               from device in devices 
               join deviceDetail in deviceDetails  on device.Id equals deviceDetail.DeviceId
               where device.Title.Contains('RaspberryPi3')
               select device;

List<Devices> deviceWithDetailsList = deviceWithDetails.ToList();

However if you have to deal with 5+ joins this means that you have some issues in application architecture, you may need to think how to simplify it or even move to NoSql database.

• Performance: Generally, it will be fast according to my example. But it depends on query complexity, expressions, aggregations, sub-queries you use alongside with joins, database (MsSql, MySql) and providers. I would recommend you create SQL query separately, perhaps in Sql Management studio (to be able to debug and see execution plan) before.
• Scalability: Architecture itself is scalable so what you need is create additional tenant shard/database (horizontal scale). In case if you need deploy application as single node and use clusters (Micro-service architecture) then you need just add application do Docker (I have plans to add Docker support) or Microsoft Service Fabric. Or you meant Joins solution ?

Would be nice if can provide some example and more details.

@AliCharper Sure, let me know if you have more questions.

[arunprasathv]

@Boriszn Thanks for the reply.

I do agree with you but the application was designed badly and we inherited it from a different team that developed a few years ago. I'm trying to optimize the performance, its relational in nature, like the customer, invoice, order entities with one to many,many to many. Without considering No SQL im trying to fix the problem. I've worked in No SQL too and the performance is better as there's no normalization and no need to worry about the index. let me know your thoughts.

Have a question regarding deviceRepository.GetAll();

1. Does it happen on the client side, does it get all records from table like a million rows to the client side and apply LINQ JOIN?
2. Is there any reason why we need to have UOW and repository pattern when EF Core itself provides that abstraction with DBContext and DBSet.

[Boriszn]

Hi @arunprasathv.

Answers:

1. No, it doesn't, it just says to db what table it should use in query. (and it returns IQueriable) Basically, my previous example just building SQL query where only line: deviceWithDetails.ToList(); executes this query.
2. In general, we need Repository to remove duplication code (DRY), separate your data access logic from business logic, etc. UnitOfWork - represents transaction (object life cycle). Avoid using this patterns you may end-up with a lot of boilerplate code, duplication, etc.

In addition to my answer:

• If you have a lot of complex joins a may need consider to use some MicroORM (like Dapper, OrmLite etc..) alongside with Entity Framework. (B/c building complex joins in LINQ to Objects maybe painful ;))
• Build, test and improve queries on DB side before you build them with EF or other ORM. (in this case you can find issues which does not related to code but to database itself)
• Always check what SQL query EF generates.

[arunprasathv]

@Boriszn Thanks for the update.

1. Also, I noticed the context is being created and not disposed of for every action, for example, latest DB updates for devices will not be fetched since the context isn't disposed of. may be notrack bheavior should be added. UOW has dispose, not sure how it disposes of device context and when it does.

2. Let's say I've 5 APIs, can I use the same context for all.

3. All Management code should be moved to a common assembly so other apis like customer,order can also use it. have a common context.

Please provide your thoughts.

services.AddDbContext(options => options.UseSqlServer(connectionString));

        services.AddScoped<IDbContext, DeviceContext>();

[arunprasathv]

@Boriszn One More thing, I do not want to use contextfactory so will change UOW as below, I just need IDBContext. Not dealing with multi-tenant architecture for now. let me know your thoughts.

private IDbContext dbContext;

public UnitOfWork(IDbContext dbContext)
    {
        this.dbContext = dbContext;
    }

[Boriszn]

Hi @arunprasathv. Thanks for the feedback.

Please find my answers below:

1. This not really true, context disposed:
   • after UoW commit was called
   • after exception in method was thrown
   • after garbage collection executed (or GC.Collect() was called).

Here code part, in UnitOfWork.cs, which does this:

.....
        /// <summary>
        /// Disposes the current object
        /// </summary>
        public void Dispose()
        {
            this.Dispose(true);

            // ReSharper disable once GCSuppressFinalizeForTypeWithoutDestructor
            GC.SuppressFinalize(obj: this);
        }

        /// <summary>
        /// Disposes all external resources.
        /// </summary>
        /// <param name="disposing">The dispose indicator.</param>
        private void Dispose(bool disposing)
        {
            if (disposing)
            {
                if (this.dbContext != null)
                {
                    this.dbContext.Dispose();
                    this.dbContext = null;
                }
            }
}
....

You can set break point debug and see (test) how it works.

2. Yes, if you gonna share context alongside with Entity Framework data models and migrations you should create assembly like YourNamespace.DataAcess and move all files from Data directory there.

3. If you have different Entity Framework models for each API then you should create assembly only from Management directory (probably create separate solution and move this library to NuGet package and share it between your APIs as NuGet dependency). Be aware that all parts of this app can be easily separated to own assembly only if it necessary (if they will reused in another projects or assemblies etc.)

4. @arunprasathv If you don't need multi tenancy please take a look on (or clone) this branch.. Everything you need, without multi tenancy feature.

[arunprasathv]

@Boriszn Thanks for the update. Have tested UoW dispose before, it worked fine for querying.

[Boriszn]

@arunprasathv Sure, your welcome.

Jus, let me know if you will have more questions. ;)

[AliCharper]

Dear Boriszn,

Can you please give us an example about using this platform in CQRS - Event sourcing. Then it would be more nice and useful.

[Boriszn]

Hi @AliCharper

Sure, I can give you example how you can use this architecture with Event Sourcing or CQRS. However, please be aware that this topic is out of scope of this Repository (and issue as well) ;-)

Options:

1. You can add additional layer (Event Query -> Handler) using Mediatr Like on 3th slide Here. Here the example repository. (You can watch the updates. So I will introduce using Data Access etc. Here the Project Board)
   2. Other option is using Message Bus as a middle-ware. You can base it on Azure Service Bus (RabbitMQ, Apache Kafka, ActiveMQ etc. ). Here the Example repository using Azure Service Bus.

Also, please take into account using following projects:

• Orleans
• Akka.Net

Hope it helps ;-)

Regards, Boris

[AliCharper]

Thanks a lot.

I have seen some of the article you have mentioned. I am going to create a full BackEND based on CQRS-ES so the Repository inside the pattern would be what you have created as an UOW sample. Will share very soon once its done.

Regards Ali

[arunprasathv]

@AliCharper @Boriszn Can't wait to see this implementation. Is there a specific use case when CQRS is a must ?

Scenarios: Back office system like billing should get data from different systems like Order Management, CRM, Customer Management etc... how would you gather data from all these systems and create Invoice finally. Data is being collected through ETL process (SSIS), can this be replaced with Message Bus, Kafka, Apache Nifi etc so we get data as it changes rather than waiting for a day.

I was thinking about CQRS as well, but it only helps in Service layer where angular calls for showing a page. Pls let me know otherwise.

What would be the best architecture you would consider for this system like this?

[Boriszn]

@AliCharper Be aware that CQRS is quite useful pattern however quite complex. (complexity in testing, understanding) before start implementing it I would recommend ask yourself: Do I really need it ? Is there other/easy way ? :)

@arunprasathv Sure there is several option how to do this:

1. Use task scheduler components like quartznet or HangfireIO Create a project/library/service (with listed components) which will work as middle ware and will constantly gather data and combine/create reports etc, from Order, Billing etc services.

2. Yes :) Use some Message Bus. (RabbitMQ, ActiveMQ, Azure Service Bus) So Billing, CRM etc will send messages to bus and Report service will gather these messages and builds the reports etc.

p.s What you think if I create Gitter/Slack chat channel, so we can move this conversation there ? And I can close this issue ?

So Here the link to the Gitter channel, now we can move this conversation there.

[Boriszn]

@AliCharper @arunprasathv Please be aware I've moved this conversation to the Gitter Room: https://gitter.im/DeviceManager-Api/Lobby. Feel free to join ;)