How to Integrate Elasticsearch in Liferay: Best Hacks and Aimprosoft’s Expertise

Key takeaways

• Elasticsearch is a part of Liferay search in enterprise projects. From this article, you will know the secrets of the Liferay Elasticsearch configuration to prevent overloads on the central server.

• Integrating Elasticsearch in Liferay requires close attention to the pre-actions the one may know before kicking off the things. Know more about them.

• We will go through the steps to deploy Elasticsearch in Liferay DXP and overview maintaining tasks with best practices from Aimprosoft.

It is essential for application developers or business analysts to ensure that all search and analysis requirements in the application are met. Integrating Elasticsearch into the Liferay project will likely become the case when you need to implement a search in a collective blog for all materials and users with tags, communities, and other things that help us categorize a massive amount of information.

In this article, we will cover the topic of step-by-step integration the number one search engine appreciated by the developer community according to DB-Engines.

Why Elasticsearch goes well with Liferay

Liferay 7 and Elasticsearch is a famous pair for powering enterprise search. First, a few words about Elastic Stack: it is a vast ecosystem of components dedicated to searching and processing data. The main components of the Elastic Stack are Kibana, Logstash, Beats, X-Pack, and Elasticsearch.

Elasticsearch is the core component of the Elastic Stack that provides a simple RESTful API, distributed systems, and uses schema-free JSON documents for data storing, searching, and manipulating. This makes Elasticsearch fundamentally different from a traditional relational database or NoSQL solutions.

Elasticsearch is a highly scalable, real-time, distributed full-text search and data analysis search engine built on Apache Lucene. The utility allows you to store, search and analyze large amounts of data. It is typically used as an underlying mechanism/technology, helping applications with complex search functionality.

The following are the main advantages of using Elasticsearch in large projects like Liferay DXP with a massive content base hierarchically structured for being accessed:

• unstructured, document-oriented;
• search capability;
• enabler of data analysis;
• support for custom libraries and REST APIs;
• easy management and scaling;
• works in real-time;
• high speed of operation;
• resilience to errors and failures.

DB-Engines’ latest usage stats dated by November 2021 show that Elasticsearch came top in the rating of database management systems by a landslide.

Search is a very important and widely used feature of Liferay application development. Liferay is a prominent Digital Experience Platform adopted by enterprises across the world that goes beyond building a traditional portal and corporate site with its customer engagement advancements.

Liferay and Elastic search go together by default across the DXP because the latter meets requirements for high scalability and full-text search typical to large collaboration platforms. In addition, Liferay benefits from the Kibana, a UI tool for indexes visualization we prefer, also referred to as the window on the Elastic Stack that is an excellent index renderer and user interface for the Elastic Stack.

Things worth paying attention to while Elasticsearch integration

As in many customization cases, be it Liferay 7 theme development or portlet creation, there are some points that have to be clear with search. Below is the Liferay Elasticsearch integration overview with checkpoints you should study before installing a standalone search engine.

Compatibility matrix

A compatibility matrix refers to certified configurations developers should follow while integrating systems. To make it right, you should be sure that the server configuration for chosen Elasticsearch version is compatible with your running Liferay version. Search Engine Compatibility Matrix introduces compatible connectors available for Liferay DXP, required patch levels, and other influencing factors that are at a place with Elasticsearch integration.

Version compatibility

It happens the supported version of Elasticsearch in Liferay DXP may not be the newest supported version. Liferay DXP does not run with Lucene as it was in Liferay 6 and lower versions. Starting from Liferay 7.1 the supported version of Elasticsearch is 6.1 or higher. Recently released (Nov 2021) Liferay DXP 7.4 GA4 is supplied with Elasticsearch 7.14.1.

It would help if you upgraded the Elasticsearch server, too, in the case of the Liferay DXP upgrade. There are securing authenticated, encrypted Elasticsearch connections. For example, to connect Elasticsearch 6 connector to Liferay DXP, you need an additional module offered in Liferay Enterprise Search on a subscription base.

On-premise and cloud setups

Behind Elastic running in the Liferay DXP search engine algorithms are embedded and are supported as a clustering. Speaking about the difference between installation on-premise setup and in the cloud, note the following.

A cloud version (and DXP Enterprise Edition) is more flexible in configuration. It is equipped with the admin panel to configure and monitor search engine settings. Configuring on-premises is more time-consuming in contrast because developers need to perform manual server configuration and setup.

Separate servers

Although Elasticsearch is embedded into Liferay, installing an additional one running on a separate remote server for production purposes reduces Liferay server load. It is reasonable because using the same JVM for Liferay and Elastic needs requires extra server consumption and creates a competitive environment for hardware resources, which is considered unfavorable. You will still be able to access embedded Elastic. Liferay SSO integration with Keycloak, by the way, goes the same way with an additional standalone setup.

Steps on how to set up Elasticsearch in Liferay DXP as it should be

In view of the foregoing, you see the importance of Liferay Elasticsearch implementation in the separate server in the Liferay DXP. Let’s go through the main steps of doing your integration perfectly. Note, extra search capabilities are feasible for DXP only (Enterprise Edition) only. A free Community Edition is supported usually by in-house Liferay developers.

Tools and software

Before downloading Elastic, be sure you’ve determined the software and tools for integration. Most often you go with:

• Linux
• Java 1.8 or higher
• Elasticsearch-7.14.x or higher

Compatibility check

Check the Liferay compatibility matrix that provides configuration details certified by Liferay. Choose Liferay Enterprise Search to find your version of DXP and the version of Elasticsearch supported by your version. It may happen you will decide to upgrade your Liferay before adding extra search capacity. For example, Liferay DXP 7.4 (released in Nov. 2021) has Elasticsearch 7.14.1 by default. That means only Elastic 7.14.x or 7.15.x (released in Sept. 2021) as a complementary search will fit your current Liferay.

It is highly recommended to check compatibility every time before you are going to install a new connector or migrate to the higher Liferay version. The Liferay and Elasticsearch versions (embedded for Elastic) have to be compatible.

Starting Liferay, you will see in logs a warning that a sidecar Elasticsearch is not recommended to use.

Follow the URL where sidecar Elasticsearch is running to see the Elasticsearch version you need.

Choosing connectors

Interconnection between systems is achieved with the help of connectors that communicate via HTTP. There is no such thing as JVM level serialization. Select a Liferay DXP Elasticsearch Connector to download before setting up an external Elasticsearch cluster for production environments. There are Java-based client or client versions such as 7.14.1 in connectors that identify the minimum compatible minor version that matches with Elasticsearch. Server connectors for communication with Elasticsearch nodes are available on the Liferay marketplace delivered by the Elastic product line vendor. The adapter configuration is carried out on the running Elasticsearch.

Elastic cluster installation

Elastic cluster installation goes along with Liferay DXP Elasticsearch configuration. You are going to create two separately tunable search infrastructures. There are two typical ways: a test setup you need to use localhost or Docker containers, in case you want to do it on your local hardware in a production environment, you’ll have to add the hosts for Liferay and the Elasticsearch cluster. Extract your downloaded copy of Elasticsearch to a folder being Elasticsearch Home on your local machine for running it. To launch Elasticsearch, you need to navigate to your [Elasticsearch Home]/bin folder for plugins activation. Command the following way:

By integrating extra search capabilities, a collection of multiple servers known as nodes is identified as a cluster. Typically it has a shared name for a cluster. The cluster can have one or several nodes that act collaboratively, sharing the workload and data.

At this stage, you can define a name for your cluster. You’re not restricted to creating something memorable and extravagant. Developers are pretty ingenious, we know, that’s for sure. Just name your cluster both in Elastic and Liferay DXP. In a code language, it may have the following view:

Specify cluster name in the [elasticsearch]/config/elasticsearch.yml file. Do not forget to save the file and then restart Elasticsearch by running the executable corresponding to your operating system. Remember, you put it in the [Elasticsearch Home]/bin folder. The start is marked with a status message, including a transport address to bind your Liferay DXP server and Elasticsearch within the network.

Ensuring security

Running Elasticsearch is expected to be secure. There are three security checkpoints: X-Pack, communication encryption, and a TLS configuration. An Elastic Stack extension is responsible for alerting, monitoring, and reporting on the issues. It is called X-Pack. By default, when you install Elasticsearch, X-Pack is installed. Typically, it is supported out of the box, however, in the case of remote Elasticsearch installation, you need to add settings in the node’s files:

Thus, you can manage users and roles in the Kibana Management app. Also, pay attention to the encrypting Elasticsearch communication by the Transport Layer Security (TLS). Generated node certificates and keys are applied to both servers, Elastic and Liferay. As for enabling a secure connection to Elasticsearch in Liferay, two options are available: doing it in the Control Panel or by a configuration file.

Configure the connection

We’re almost set up Elasticsearch in Liferay. There’s only time to configure the connection between Liferay DXP and Elasticsearch cluster. Follow the path with started DXP:

Control Panel → Configuration → System Settings → Search → Enter ‘elasticsearch’

Here you can configure the following options:

• Cluster name (you created in Elasticsearch);
• Operation mode (override EMBEDDED to REMOTE);
• Transport address (address from the Elasticsearch server).

Great, now it’s time to save on.

Reindexing

Now stop it. Yes, you haven’t heard wrong. After backing up the DXP, restart Liferay and reindex all search indexes in “Index Actions”. Reindexing reduces the space consumption of the index. New index versions prevent dead pages and failures related to incompatibility between indices and nodes. Follow the path:

Control Panel → Configuration → Search → Execute

There will be shown some messages you should scroll up in the Elasticsearch log.

Conclusion

As you see, it is not a big deal to integrate the standalone search engine for content management platforms to enhance its capabilities by following a clear non-boring guide. If you still have some questions, contact us to clear them up. We every day cope with Liferay development tasks, so one of your concerns most likely is on our radar.

FAQ