Precompiled Razor Views

Sun Feb 2, 2020 in Development , Sitecore using tags Sitecore , Performance , ASP.NET MVC , Razor

Precompiled Razor Views - Sitecore 9 - Issue 192470 & 119596

There are already some posts on this topic of using Precompiled Razor Views, so I won’t duplicate that content.

Kam’s blog - Precompiled Views supported by Sitecore since Sitecore 8.2

Jeremy’s blog - possible side affect of Precompiled Views

Chris’s blog - Slow compilation when using MVC areas/Helix folder structure

Matt’s blog - ensuring your razor view paths are unique

And probably others, but these one’s stood out to me, and relevant for this post.

Read these articles first, I’ll wait.

Quick summary

So you’ve read the above articles, here’s a quick recap:

  • we know Precompiled Razor Views are supported by Sitecore since Sitecore 8.2.

  • We want to measure/benchmark before and after applying this performance fix, how else would you know it’s had the desired effect. And watch out for this Disk IO calls, which seem to be occurring even if have the Sitecore setting disabled “UsePhysicalViewsIfNewer” which seems counterintuitive.

  • If you are following a MVC areas/Helix style structure with multiple folders for your views, it looks like using precompiled razor view is going to be really beneficial in your case.

  • Ensure you are using unique MVC area/Helix folder names & unique razor file paths across projects to avoid issues, just like you would if they were physical files in one project.

Ok - I’ve read the above articles, what more are you writing about?

Well first I want to draw attention to “UsePhysicalViewsIfNewer” setting.

Check in your Sitecore setup what this value is set to.

If for example you have received a Sitecore Support patch/hotfix with a new .cshtml file

E.g. Issue 214239 for “\sitecore\shell\client\Business Components Library\version 2\Layouts\Renderings\ListsAndGrids\Grid.cshtml”

Which includes a config patch which changes “UsePhysicalViewsIfNewer” to true to load the .cshtml file with the fix, rather than the precompiled view in the assembly.

Those disk IO calls which seem to occur either way (more on that later), are defiantly going to occur if you tell it to.

This also explains the guidance you might see where the .cshtml file last modified data must be more recent than the .dll file, otherwise it wouldn’t be loaded.

So possibly a Sitecore Support patch to a .cshtml file might undo some of the performance gains we are trying to achieve. As with everything measure.

If you don’t have “UsePhysicalViewsIfNewer” set to true, can skip this next section, and skip to the hotfix, otherwise read on.

I want a support fix and performance settings

Disclaimer, I’ve only tried this on a dev machine. And not completed benchmarking/testing

Maybe if you are only worried about the Sitecore MVC .cshtml hotfix for Content Authoring, and only want the compilation benefit on Content Delivery so could change this setting “UsePhysicalViewsIfNewer” to true just for Authoring?

However, if you want to have the compilation benefit on Content Authoring as well as the MVC .cshtml fix. One possible way could be to create a new project named “ZZZZZ…” with Razor compilation enabled and place the fixed .cshtml file in that project with the matching path of the file you want to override from Sitecore.

And add to the pre-compilation configuration a reference to this new assembly.

<configuration xmlns:patch"http://www.sitecore.net/xmlconfig/">
    <sitecore>
        <mvc>
            <precompilation>
                <assemblies>
                    <assemblyIdentity name="ZZZZZ......AssemblyName" />
                </assemblies>
            </precompilation>
        </mvc>
    </sitecore>
</configuration>

Why call the project ZZZZZ….

We are going to turn the bug from Matt’s blog - ensuring your razor view paths are unique into a solution.

“The problem becomes apparent when two assemblies have the same generated virtual view path, the assembly that comes alphabetically last wins out and overwrites the value corresponding to the virtual path key string in the _views dictionary.”

We are going to deliberately overwrite the Sitecore Pre-compiled view, with the fixed Pre-compiled view, but naming the assembly with the fixed .cshtml file with a name Alphabetically after the original Sitecore Assembly.

If this is a good solution without drawback, perhaps support could do this, to save us the time of doing it ourselves?

Or actually Sitecore could re-issue the original .dll with the fixed Razor view, so don’t need to overwrite it.

And now with the new Hotfix only model, they probably would do this.

Still this might be relevant for an existing patch you have, prior to the new Hotfix model.

Best of both

With this technique, can leave “UsePhysicalViewsIfNewer” set to false, and override a Sitecore precompiled view.

More testing required to verify this works without drawback.

Great so I’ve definitely got “UsePhysicalViewsIfNewer” set to false, what’s this Hotfix

Disclaimer, I’ve only tried this on a dev machine. And not completed benchmarking/testing

Going back to Jeremy’s blog possible side affect of Precompiled Views. If look at the comments section, can see I enquired about if this was still an issue in Sitecore 9, and had a little more success from Sitecore Support who confirmed this was a bug in Sitecore and not the RazorGenerator project.

Looking at the contents of Sitecore.Support.192470.dll, can see it removes the first MVC view engine from the list, I asked Sitecore Support why and got this answer:

The setting UsePhysicalViewsIfNewer works as expected for all precompiled views except the ones that come from a couple of SPEAK dlls (Sitecore.Speak.Web.dll and Sitecore.Speak.Components.Web.dll). The UsePhysicalViewsIfNewer setting value is used to initialize PrecompiledViewAssembly objects which will be a part of the RazorGenerator.Mvc.CompositePrecompiledMvcEngine. Sitecore initializes an instance of Sitecore.Mvc.SitecoreRazorViewEngine derived from the CompositePrecompiledMvcEngine. And further the instantiated engine is encapsulated in the Sitecore.Mvc.InstrumentendViewEngine type.

So far so good.

The problem is that there is an extra PrecompiledMvcEngine instance that comes from SPEAK. This particular engine instance ignores the UsePhysicalViewsIfNewer setting. The engine instance gets placed at the very first position of the ViewEngines.Engines list. And the issue happens when this “incorrect” engine is taken via a FirstOrDefault LINQ func by Sitecore MVC. Please also notice that with the current architecture the only possible way to fix this is to remove the engine from the list.

This would seem to explain the issue Jeremy was having.

However I’ve yet to benchmark this solution, to see if solves the disk IO issue, but got a logical explanation from Sitecore Support.

I’m not sure if this is still an issue or fixed in later version of Sitecore after Sitecore 9. But a quick scan through the recent version release notes I can’t see either of the public reference numbers 192470 or 119596.

Summary

Disclaimer, I’ve only tried this on a dev machine. And not completed benchmarking/testing

If you are experiencing issues with Compiled Razor views and disk activity even with “UsePhysicalViewsIfNewer” set to false, perhaps check if this fix is applicable for the version of Sitecore you are on, and measure if it has the desired impact.

Measure Everything

Sitecore MVP 2020 - Technology

Sat Feb 1, 2020 in Development , Sitecore using tags MVP , Sitecore

Sitecore MVP 2020 - Technology

I’m thankful to receive my 1st Sitecore MVP award in the Technology category for 2020.

MVP MVP

I’ve been using Sitecore since Sitecore 6 in 2011.

Keep sharing and trying if didn’t make it this year, this is my first year I’ve been fortunate enough to be selected for the MVP award after several years of applying.
Although it’s not enough to just keep on applying, you can’t get the award if you don’t apply, and you might get some feedback what you need to improve upon.

A lot of my activity with Sitecore might not be publicly visible, and probably explains why I’ve not received this award previously.
Digging out dotpeek, finding the root cause of an issue, and possibly supplying a fix on occasion to Sitecore support for review (of course would always prefer to use an official hotfix, but to prove have found the problem & working solution I might get there first).
Or again using dotpeek finding a way to add in support for a feature that isn’t currently supported, and sharing the approach with Sitecore to make it easier to extend in future upgrades, or be integrated into the product in the future.
This might not be visible to the community straight away, but hopefully makes the product better in the long run.

Now with the new single Hotfix model per version of Sitecore, these identified issues fixes are more likely to get into the product quicker, as well as making upgrades easier.

I’ve been trying to share what I’m up to, either blogging, asking/answering questions on Stack Exchange, or presenting when I can.

I’ve also been on a train the trainer course on .net core, Docker, K8s and AKS, and delivered this training.

As well as attended a course on improving my public speaking skills.

All of which has helped me practice my public speaking.

Special mentions to Sitecore User Group UK and Sitecore Discussion Club for letting me present.
And Steve McGill and Martin Miles for running them.

As well as to the Sitecore Support & Product teams for collaborating on tickets, and working on getting the fixes/enhancements into the product.

Plus of course the awesome Sitecore community, whether that’s Twitter, Sitecore Slack, Blogs, Sitecore StackExchange, SCUG Meetups & Sitecore Discussion Club, SUGCON, Symposium etc.

As well as the awesome team I work with, but as this is my personal blog I’ll leave it at that.

Thanks for reading my blog, more to follow!

Improving the Siteore Solr Content Search Provider

Tue Nov 26, 2019 in Development , Sitecore using tags Solr , Sitecore

Improving the Siteore Solr Content Search Provider

Back in February this year I gave a presentation at the London Sitecore User Group “Improving the Solr Content Search Provider”.

My first talk at a Sitecore User group.

Although I regularly attend the Sitecore discussion club, and have presented there on several occasions.

Known bugs

At the presentation I included a link to a github repository with a list of known issues and their public reference numbers.

The list of issues will vary between Sitecore versions, I’ve included the list of issues I’ve come across between Sitecore 8.2 up to Sitecore 9.0 update 2.

Including:

  • Partially rebuilt index going live - 96016 - IIS recycle - switches Alias although indexing job cancelled/incomplete - fixed in Sitecore 9 initial release
  • OptimizeOnRebuildEnabled setting not used - was in bug fix 96016 - introduced in Sitecore 9 initial release
  • Patch for IsSolrAliveAgent to update SolrState and process reinitialisation correctly - 163850.171950
  • Incorrect data being indexed, if have ContentSearch.Indexing.DisableDatabaseCaches setting set to true - 96740.127177.155383
  • If IndexAllFields=false the IncludedFields are indexed as string values - New bug in Sitecore 9 - 252532
  • Index rebuild slow down after Sitecore 9 upgrade.

Common Patterns for distributed computing

The github repostiory also included an example implementation of patterns to address some of the fallacies of distributed computing.

  • Circuit Breaker Pattern
    • Don’t make every request wait for a timeout exception, use a circuit breaker to fail fast, so can handle load under a degraded scenario
  • Not swallowing errors
    • We want to know the difference between no results, and an error.
    • The circuit breaker can’t work if we don’t know what’s an error.
    • If we don’t know the difference then we might cache there are no results, when there are results, but just couldn’t retrieve the results.
  • Shorter timeouts for queries
    • Again, we want queries to fail fast, and for the circuit breaker to kick in. But we can allow longer for indexing operations to take place, especially as there is no retry mechanism on the crawling/indexing side.

Comparison of the other options out of the box

  • Sitecore Query
  • Sitecore Fast Query
  • Links Database

Related blog posts:

On reasons not to use each of them, at least on Content Delivery, or when querying over lots of items.

A list of other alternatives

Examples of Customisations

Changing MyItems & Unlock all to use Solr, rather than a Sitecore Query.

I told you Sitecore.Query was slow, and Memory Intensive.

Since giving this presentation

Infact since giving this talk, one of the issues that got me to customise the Sitecore Solr Search provider in the first place reared its head again. Sitecore not being able to startup when Solr is down.

I thought we’d fixed that already, why can’t Sitecore startup when Solr is down anymore?

Well the number of Sitecore indexes had grown over time.

On startup each index was doing it’s own check to see if Solr was available, and waiting for a timeout/exception. This was happening in series. Depending on the type of error you get, if it’s a timeout, this can result in Sitecore taking longer to startup than IIS allows. Causing IIS to recycle the application and try again, in an infinite loop. Oh dear!

So I’ve added to the github repository a new issue 314454 - Sitecore doesn’t startup when Solr Down.

Simulating Solr being down/latency

You can simulate on your local PC latency to Solr using Fiddler. If you change your solr connection string to

http://localhost.:8983/solr;solrCloud=true

Notice the “.”, will ensure the traffic goes through Fiddler

Change your Sitecore IIS Application Pool to run as your local user, as Fiddler automatically picks up traffic running as your local user. (Or you can change the proxy settings)

Then run Fiddler.

Under “AutoResponder” tab, you can add a Rule for requests matching

http://localhost.8983/solr/.....

The URL you set could be specific to a particular index, or request, depending on what you want to test.

Then on the response for that URL you can set

*delay:10000

with how many milliseconds you want to delay the request with.

Using this technique you can simulate a network timeout, rather than a quicker port not listening/service not running error.

And from this can replicate the issue of Sitecore note being able to start up, if you have enough indexes which all timeout on startup.

What’s in 314454 - Sitecore doesn’t startup when Solr Down

Sitecore doesn’t issue patches anymore, so instead you get a hotfix, which include all of the issue fixes for that version of Sitecore you are on.

For Sitecore 9.0 update 2, this was included in hotfix 323662-1.

  1. If you have enough indexes, the timeout on initialisation from each index run in sequence can result in Sitecore not being starting up in the allowed time.
  2. Retry logic for SolrCloud aliases.
  3. Retry logic for initialising Indexes
  4. Exception handling in IsOnline index check.
  5. Initialisation of indexes to not be interupted if Solr is unavailable, to initialise what can and retry later.

And included changes to dlls

  • Sitecore.ContentSearch.Client.dll
  • Sitecore.ContentSearch.Data.dll
  • Sitecore.ContentSearch.dll
  • Sitecore.ContentSearch.Linq.dll
  • Sitecore.ContentSearch.Linq.Lucene.dll
  • Sitecore.ContentSearch.LuceneProvider.dll
  • Sitecore.ContentSearch.SolrNetExtension.dll
  • Sitecore.ContentSearch.SolrProvider.dll

There were so many changes it felt like a mini Sitecore Upgrade. And the contract changed, had to rewrite our customisations for them to continue to work.

This hotfix fixes the slow startup issue, but only checking once if Solr is available, rather than per index. So make Sitecore much faster to startup when Solr is down.

There was also some work to make the initialising of the indexes to work in parallel, to further speed up the time for initialisation.

Customisations and Upgrades

Each time we upgrade, we have to see if our customisations are still possible. Often the extension points/hooks we have used have gone, and we have to find new ones.

Even on this update to the hotfix which address 314454, we had to do a lot of rework.

Hopefully in a future version of the product the extensions points we require will be included in the product, and we won’t have to get out dotpeek to find a place to override, and use reflection/so many custom classes and overrides to change the behaviour to what we need.

Update the repository examples

As the hotfix isn’t publicly available in the nuget feeds, and the contract has changed.

I can’t update the example code to reference the hotfix, and for it to still compile.

If anyone is interested to see the examples updated to use the hotfix, reach out to me on twitter, and I can see about creating a feature branch.

Otherwise, I’m going to wait for a version of Sitecore which includes these fixes/contract changes, which is available in the nuget feed so I can update the examples and still have it compiling.