Switch Sitecore 9 From Azure Search to Solr -- Step-By-Step Instructions

In Sitecore 9 you can use your choice of search providers; Lucene, Solr or Azure Search. When you deploy Sitecore XP to Microsoft Azure using the Marketplace Module, it configures Azure Search for you by default. But what if you want to use Solr instead?

There are several reasons you might want to do this:

  • Azure Search is limited to 1000 fields in the index. This is fine for Sitecore XP out-of-the-box, but if you start adding modules like SXA, Forms, EXM and Commerce, you will exceed the 1000 field limit and you won't be able to rebuild your index. You can work around this problem by changing your index configurations to decrease the number of fields being indexed, but this can be complex and there's no documentation on which fields may be excluded. Sitecore says they will address this problem in the 9.1 release.
  • Azure Search can be expensive to operate. In our experience running a "Small" sized environment in Azure, the Azure Search service was by far the most expensive component. And there's no way to turn it off when you're not using it.
  • To my unscientific observation, Solr seems to perform much better than Azure Search. Index rebuild times are greatly reduced.

I could find no documentation on how to switch Sitecore 9 on Azure to use Solr, so I powered my way through it and made all the mistakes so you don't have to. There were a few challenges along the way, the main ones being that xConnect will not work with Solr if you're using self-signed certs, so you'll need to purchase a domain name and SSL cert from a Certificate Authority. And the other challenge was poking around all the App Services to find the configs that needed to be updated.

But now it's up-and-running, and works like a charm. Here's an overview of what I did to make it work.

  • Create a new virtual machine in Azure to host your Solr instance
  • Install Solr on the VM
  • Purchase a domain name and set up an Azure DNS Zone
  • Purchase an SSL certificate from a CA and install in Solr
  • Update Sitecore config files
  • Test connections, populate schemas and rebuild indexes

 Ready to get started? Let's do this thing.

1. CREATE A NEW VIRTUAL MACHINE IN AZURE

a. Log in to your Azure subscription, click "Create a resource", and select Windows Server 2016 VM.

b. Configure basic settings for your VM and click "OK":

c. Choose a size for your VM. It's best to start small and only scale up if you really need to.

d. Configure setting. You can leave them all at the default setting, but make sure you create a public IP address. You'll need this for SSL later. Click "OK" to continue.

e. Review the summary screen and click "Create" to create your new VM. It will take a while for Azure to spin up your VM.

f. Navigate to your new VM resource, go to Overview, and click the Connect button. Download your RDP file so you can remote connect to the VM.

g. Now you can RDP into your new VM and continue the process.

2. INSTALL SOLR

a. Install the Java runtime environment on your VM. https://java.com/en/download/

b. You have to add 2 new environment variables. Click the search icon, and type in "environment". Click the "Edit the system environment variables" app.

c. Click the "Environment Variables" button.

d. Add two new system variables, JAVA_HOME, and JRE_HOME, point them to the root of the Java JRE you just installed.

e. Download solr-6.6.2 and unzip the files to C:\Program Files\solr-6.6.2\:
https://archive.apache.org/dist/lucene/solr/6.6.2/solr-6.6.2.zip.

f. Make sure Solr will start by running the following in a command prompt with administrative privileges:

C:\Program Files\solr-6.6.2\bin\solr.cmd -p 8984

Once successfully started up, you should be able to navigate to http://localhost:8984 to make sure that the instance is running. If no errors are encountered and Solr started successfully, stop Solr by running the command:

C:\Program Files\solr-6.6.2\bin\solr.cmd stop -all

g. Now you can set up Solr to run as a Windows service.

Download and extract Non-Sucking Service Manager (NSSM) from https://nssm.cc/download. Run the following in command prompt with administrator privileges. Update path to where you extracted nssm:

"C:\nssm-x.xx\win64\nssm.exe" install SOLR6

Set the path to 'C:\Program Files\solr-6.6.2\bin\solr.cmd' and Arguments to 'start -f -p 8983':

Finally, start the SOLR6 windows service and navigate to http://localhost:8983 to make sure Solr is running.

3. PURCHASE A DOMAIN NAME AND DELEGATE TO AZURE DNS

If you're running production servers you already have a domain name, but if not, you will need a public domain name for your SSL certificate. There are lots of cheap options for this (less than $10 for a year). I personally use GoDaddy because they have $0.99 domain names, good support and it's easy to use their website.

a. Go purchase a domain name from your choice of providers.

b. Delegate your domain to Azure DNS. There are good instructions for this here: https://docs.microsoft.com/en-us/azure/dns/dns-delegate-domain-azure-dns. You will create your new DNS zone in Azure, and retrieve the list of Azure name servers. You then need to update your domain provider with the name servers.

4. PURCHASE AN SSL CERTIFICATE AND CONFIGURE SOLR TO USE IT

Again, if you're running production servers you may already have SSL certs you can use. But if not, follow these steps to create one. (There are cheap SSL certs available for under $5/year. Google it.)

a. The first step is to create your certificate signing request, (CSR). Fire up Powershell, and change directory to "C:\Program Files\solr-6.6.2\server\etc". Then run the following PowerShell commands after filling in the parameters:

<JRE PATH> - The path to your Java installation root folder, for example "C:\Program Files (x86)\Java\jre1.8.0_171".
<YOUR PASSWORD> - This should be a strong password because it is used to secure your keystore. Come up with a good password and make note of it.
<DOMAIN> - Your fully qualified domain name, for example "solr.contoso.com".
<IP ADDRESS> - The public, static IP address of your Solr VM server.
<ORGANIZATIONAL UNIT> - Your OU, for example "Dev" or "Sales".
<ORGANIZATION> - Your company name, for example "Contoso Company".
<CITY> - Your company's city.
<STATE> - Your company's two-letter state. For example "VA".
<COUNTRY> - Your company's to letter country, for example "US".

-- Create the keystore --

& '<JRE PATH>\bin\keytool.exe' -genkeypair -alias solr-ssl -keyalg RSA -keysize 2048 -keypass <YOUR PASSWORD> -storepass <YOUR PASSWORD> -validity 9999 -keystore solr-ssl.keystore.jks -ext SAN=DNS:<DOMAIN>,IP:<IP ADDRESS> -dname "CN=<DOMAIN>, OU=<ORGANIZATIONAL UNIT>, O=<ORGANIZATION>, L=<CITY>, ST=VA, C=US"

-- Create the CSR --

& '<JRE PATH>\bin\keytool.exe' -certreq -keyalg RSA -alias solr-ssl -file <DOMAIN>.csr -keystore solr-ssl.keystore.jks -ext SAN=DNS:<DOMAIN>,IP:<IP ADDRESS>

Now you will have a file name <DOMAIN>.csr which you can submit to your CA to generate the SSL certificate. Once the CA sends you your certificate files, proceed to the next step.

b. If your CA gives you the option to select which format you want to download your certificates in, select Tomcat. This will provide you with three files; a bundle cert, your personal cert, and a .pem file. You need to package the two .crt files together into a new .crt file. So create a new, empty text file named <DOMAIN>.crt, open it, and paste in the contents of your bundle .crt file. Then paste in the contents from your cert file. Make sure there are no spaces or line breaks anywhere, then save the file.

c. Now you will import your certificate into your keystore. Use the following PowerShell command:

& '<JRE PATH>\bin\keytool.exe' -import -alias solr-ssl -keystore solr-ssl.keystore.jks -trustcacerts -file <DOMAIN>.crt

d. Create a .P12 file that you can use to register the certificate in Windows:

& '<JRE PATH>\bin\keytool.exe' -importkeystore -srckeystore solr-ssl.keystore.jks -destkeystore solr-ssl.keystore.p12 -srcstoretype jks -deststoretype pkcs12

A file named <DOMAIN>.P12 will be created. Double-click the file and the certificate import wizard will start. Follow the wizard to install the cert in 











e. Time to configure Solr to use SSL. Edit the file "C:\Program Files\solr-6.6.2\bin\solr.in.cmd", and uncomment all the properties starting with Solr_SSL_*, like so. And make sure to put in your keystore password:

REM Uncomment to set SSL-related system properties
REM Be sure to update the paths to the correct keystore for your environment
set SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.jks
set SOLR_SSL_KEY_STORE_PASSWORD=<YOUR PASSWORD>
set SOLR_SSL_KEY_STORE_TYPE=JKS
set SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.jks
set SOLR_SSL_TRUST_STORE_PASSWORD=<YOUR PASSWORD>
set SOLR_SSL_TRUST_STORE_TYPE=JKS
set SOLR_SSL_NEED_CLIENT_AUTH=false
set SOLR_SSL_WANT_CLIENT_AUTH=false

f. Since you are using your own password, and not the default "secret" one, you'll also need to update the jetty-ssl.xml file. Navigate to "C:\Program Files\solr-6.6.2\server\etc\" and open "jetty-ssl.xml" for editing. Find the two nodes below and edit them to add your password:

<Set name="KeyStorePassword"><YOUR PASSWORD></Set>

and 

<Set name="TrustStorePassword"><YOUR PASSWORD></Set>

g. Start "Windows Firewall with Advanced Security", and create a new Inbound Rule named "solr_inbound". Configure the rule as follows:



h. Lock down the server so that only selected remote IP addresses can reach the server. This should include all your Sitecore app services, and any developer machines that need to access Solr.

h. Create a new Outbound Rule named "solr_outbound", and configure as follows:



i. Open Notepad with the "Run as Administrator" command, and open your hosts file for editing. The host file lives in "C:\Windows\System32\drivers\etc". Add the following entries to the bottom of the file with the parameters replaced with your values:

127.0.0.1 <DOMAIN>
<IP ADDRESS> <DOMAIN>

i. Let's see if our new https connection works! Restart Solr, and navigate your browser to "https://<DOMAIN>:8983/solr". If all goes well, you should see your Solr admin screen. Try it from a different computer to make sure the Solr server can be connected to from other servers.

5. UPDATE SITECORE CONFIGURATIONS

We need to reconfigure Sitecore App Services to use Solr instead of Azure Search.

a. Update <root>\web.config files on the following server roles:

  • CM
  • CD
  • Prc
  • Rep

Change this node:

<add key="search:define" value="<Azure or Lucene>" />

...to this...

<add key="search:define" value="Solr" />

b. Update <root>\App_Config\Sitecore\ContentSearch\Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config on the following server roles:

  • CM
  • CD
  • Prc
  • Rep

Update these nodes:

<setting name="ContentSearch.Solr.ServiceBaseAddress" value="https://<YOUR SOLR URL>:8983/solr" />
<setting name="ContentSearch.SearchMaxResults" value="500" />
<setting name="ContentSearch.Update.BatchModeEnabled" value="true" />
<setting name="ContentSearch.Update.BatchSize" value="500" />

 

c. On the following server roles:

  • MA-OPS
  • MA-REP
  • xc-Collect
  • xc-Search
  • xc-Refdata

Disable this file: <root>\App_data\config\sitecore\CollectionSearch\sc.Xdb.Collection.IndexReader.AzureSearch.xml
Enable this file: <root>\App_data\config\sitecore\CollectionSearch\sc.Xdb.Collection.IndexReader.SOLR.xml
Enable this file: <root>\App_data\config\sitecore\CollectionSearch\sc.Xdb.Collection.IndexReader.SOLR.xml

Disable this file: <root>\App_data\config\sitecore\SearchIndexer\sc.Xdb.Collection.IndexWriter.AzureSearch.xml
Enable this file: <root>\App_data\config\sitecore\SearchIndexer\sc.Xdb.Collection.IndexWriter.SOLR.xml

d. On the following server roles:

  • xc-Search

Disable this file:
<root>\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\CollectionSearch\sc.Xdb.Collection.IndexReader.AzureSearch.xml
Enable this file: <root>\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\CollectionSearch\sc.Xdb.Collection.IndexReader.SOLR.xml
Enable this file: <root>\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\CollectionSearch\sc.Xdb.Collection.WebClient.SOLR.xml

Disable this file:
<root>\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\SearchIndexer\sc.Xdb.Collection.IndexWriter.AzureSearch.xml
Enable this file:
<root>\App_data\jobs\continuous\IndexWorker\App_data\Config\Sitecore\SearchIndexer\sc.Xdb.Collection.IndexWriter.SOLR.xml

e. Update<root>\App_Config\ConnectionStrings.config on the following server roles:

  • xc-Search

Disable this connection string: <add name="collection.search" ... />
Add a new connection string: <add name="solrCore" connectionString="<URL TO YOUR SOLR INSTANCE>:8983/solr/xdb/"/>

NOTE: I know the Url above doesn't resolve to anything, but it needs to be formatted this way. Do not add any # in the Url path.

f. Update<root>\App_data\jobs\continuous\IndexWorker\App_config\ConnectionStrings.config on the following server roles:

  • xc-Search

Disable this connection string: <add name="collection.search" ... />
Add a new connection string: <add name="solrCore" connectionString="<URL TO YOUR SOLR INSTANCE>:8983/solr/xdb/"/>

NOTE: Again, the Url above doesn't resolve to anything, but it needs to be formatted this way. Do not add any # in the Url path.

6. CREATE THE SOLR CORES

Now that we have connectivity to Solr from all our App Services, we can follow these steps to populate the Solr schemas:

a. Copy the Solr provided configuration example from the configsets folder into [SOLR_DIR]/server/solr and rename it to the appropriate index name.

b. Open the managed-schema file under the conf folder of folder created in step a.

i. Set the value in <uniqueKey>id</uniqueKey> to _uniqueid.

ii. In the fields section, add field configuration for _uniqueid:

<field name="_uniqueid" type="string" indexed="true" required="true" stored="true"/>

iii. Save your changes.

c. Repeat step 1 and step 2 for all indexes (DO NOT create the analytics index yet - we'll do the a bit later).

d. Enable Solr Term support.  When you implement Solr for use with Sitecore, you must enable term support in the Solr search handler. The term functionality is built into Solr but is disabled by default. To power the dropdowns in the UI, you must enable the terms component. To enable Solr term support:

i. For each core you have created, open the solr-config.xml file under in conf folder. Locate the requestHandler node:
<requestHandler name="/select" class="solr.SearchHandler">. Inside the requestHandler node, you need to add the following line:

<bool name="terms">true</bool>

ii. Under the "defaults" node, add the following line:

<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
<str name="df">text</str>
<bool name="terms">true</bool>
</lst>

iii. After the "defaults" element, add the following node:

<arr name="last-components">
<str>terms</str>
</arr>

d. Start Solr.

e. Start Sitecore, and go to the Control Panel. In the Indexing tab, click "Populate Solr Managed Schema":

y

g. In the Schema Populate window, select the indexes you want to populate:

b

h. Click "Populate".

i. Copy the Solr provided configuration example from the configsets folder into [SOLR_DIR]/server/solr and rename it to "xdb". No further updates are required. This is your primary xDB index. 

j. Repeat step i., and name this one "xdb_rebuild". No further updates are needed. This is your secondary xDB index.

7. REBUILD SITECORE INDEXES

Now you are ready to rebuild your main Sitecore indexes in the usual manner.

8. REBUILD xDB INDEXES

Rebuilding the xDB indexes is a bit trickier. 

a. Log in to your Azure subscription, and navigate to you xc-search App service.

b. Click "Advanced Tools", and then click "Go".

c. In the Kudo Services page, click to open the CMD Debug Console.

d. Change directory to get to D:\home\site\wwwroot\App_data\jobs\continuous\IndexWorker.

e. Type "xConnectSearchIndexer -rr" and hit enter to start the rebuild. When it's finished you should see a screen like this:

And guess what? You're all done! It was a long slog to get there, but now you are up-and-running with Solr on Sitecore XP Cloud.

One last note: now that you are running Solr in a VM, you can stop the VM when you're not using it to stop the billing. When you need it again, just fire it back up and off you go!

As always, I welcome any comments or thoughts about how to improve this blog, or my coding and techniques.

~David

Comments (2) -

  • Outstanding Article! Thank you for sharing.
  • Hi David.It is a Great Article. However I would like to know that what steps we need to follow to migrate from Solr to Azure Search ?

Add comment

Loading