This project is read-only.

Scalability problem caused by cascading

Jan 11, 2009 at 1:14 PM
Hi,
Problem:
When saving or deleting a page, NHibernate cascade functionality runs on all siblings (pages next to each other on the same level). If you have a couple of thousand sibling pages and decides to add a new sibling, all siblings will be saved. Which in reality means you can't have many siblings because the cascade functionality eats up all server power.

The reason I have dug into this problem is that I really want to use N2, but with many siblings. Do you have any suggestion on how to solve this? I can think of changing the cascade options (in nhibernate mappings default.hbm.xml) from "all"/"all-delete-orphan"  to "delete-orphan", which I think means it only deletes the childs if the parent is deleted. But will this affect N2? I appreciate any thoughts related to this problem!

Best regards
Jan 11, 2009 at 5:51 PM
Any idea why the siblings are beeing cascaded? Is it possibly related to the sort order beeing updated? I did try tweaking the cascading option for children as per your suggestion but this seem to trigger an exception "no collection snapshot for orphan delete". I must admit I don't really understand what's going on. My assumption was that all would save and update changed children in addition to delete-orphan's cascade deleting. I'll try to reproduce the behaviour and see what cause the cascade.


Jan 13, 2009 at 7:51 PM
Thanks for your fast reply.

I have now found the problem areas but needs some help with identifying the purpose of some functions in order to come up with a scalable solution. The cascade theory did not have a big impact on scalability. So consider that statement as false.
The problem areas I found were as follows:

- N2.Integrity.IntegrityManager.IsLocallyUnique. It traverses all siblings. A scalable solution is to replace this with a simple database query, asking for a content item with the specific name.

- N2.ContentItem.AddTo. It traverses many(all?) siblings. Here I need help identifying the purpose of the method. Particularily between line 489 and 505 (latest source release).

- N2.Persistence.NH.ContentPersister.EnsureSortOrder. It traverses alla siblings. Here I would also need help with identifying the purpose of this method.

I did the benchmarking tests with 11000 siblings. The result was that a creation of a page took 300 times longer with 11000 siblings than with 10 siblings. By removing the above mentioned methods from the saving session, creation time was nearly identical with 11000- and 10 siblings. I also did benchmark on the nhibernate cascade functionality which proved to only increase the creation time by 100% no matter the amount of siblings (which means the scalability ins't affected).

Looking forward to some help!
Jan 14, 2009 at 7:33 PM
Good suggestion about the IsLocallyUnique.
The stuff in AddTo is a way to keep the trashcan at the bottom of the list.
The EnsureSortOrder is used to update the sort order of newly added (or moved) items so the same order is applied when retrieving children from the database next time. Do you have a suggestion for this case? Select max?

Do you know if the loops triggers any lazy database queries? That could be a really bad bottleneck. How did you profile the code? Can you post any details?
Jan 14, 2009 at 9:52 PM
AddTo: So this functionality should only run with siblings having the root page as parent? If that is the case a solution is to just to select the sibling with the highest sortorder and add 1.
EnsureSortOrder: A solution here would then be to select the sibling with the highest sortorder and add 1, if the page is created. If the page is updated (moved) the current functionality should be run.

I will go ahead and implement these fixes if you agree with my solutions.

I don't know about the lazy loading, the only lazy loading I find is when the ContentItem.Children property is accessed. I profiled the code by inserting log messages for start and stop to determine the run time of a code section. Nothing special, just enough to get what I wanted.

Jan 15, 2009 at 11:20 PM
I'll agree to a solution when I see it =) You seem to know what you're doing so I'm sure that's not a problem.
Jan 29, 2009 at 10:00 PM
Just wanted to tell you that I have implemented new versions of the above mentioned methods. The problem is I had to change some logic, specifically in AddTo where I removed the trashcan handling and just saved a new page at the end of the page list. The result is what I wanted though: possibility to add a page with many siblings without huge performance losses. I also did code review and tests on page delete functionality and found similar scalability problems. My advice to you, if you want the platform to handle larger amounts of page data in the future, is to work more directly against the database rather then performing filtering and querying against object lists. If you want the code I will e-mail it to you, just let me know!