Elasticsearch Using Nest

Elasticsearch is a search server based on Lucene. It provides a distributed, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents. Elasticsearch is developed in Java and is released as open source under the terms of the Apache License.

Background information about Elasticsearch Wiki.

To understand Elasticsearch, try the following link : ElasticSearch

In this blog, I will try to explain how we can use Elasticsearch in .net using NEST.

I have created the same application using visual studio 2013. You can download it from here.

Nest is a high level client that provides a strongly typed query DSL that maps one-to-one with the Elasitcsearch query DSL, and takes advantage of specific .NET features such as co-variant results. NEST internally uses, and still exposes, the low level Elasticsearch.Net client.

Elasticsearch provides API’s to build index and to implement text search.

Why we need to use Elasticsearch:

Just have a look over the below mentioned SQL Query.

In the above example, I am trying to join 4 tables in order to fetch required data. Meanwhile we need to take care of not having duplicate rows in the result. Let’s assume that table has around 10,000 rows and then this query will definitely hits the performance of the application. So it’s better to execute these queries and pull the results before hand and index these documents once. Later while searching, search over these index files and let the SQL to handle other operations. For more information watch this video.

Set Up:

1. Download the latest version of Elasticsearch from here.
2. Unzip it and copy it to local folder.
3. Now we need to set JAVA_HOME variable.
4. In C drive, within program files folder there will be a JAVA folder and within that folder jre folder.
5. Open that folder and copy the folder path.
6. Now open the command prompt and type as mentioned below

set JAVA_HOME=paste the folder path you copied earlier

7. Then press enter. It will set the JAVA_HOME variable.
8. Now in command prompt, redirect to the unzipped Elasticsearch bin folder.
9. After redirecting, type in command prompt as mentioned below

elasticseach

10. Then press Enter.
11. This will activate the Elasticsearch.
12. If everything goes right, you will get something like as shown in below image

13. To check, elasticsearch is working properly, just open "http://localhost:9200/" in browser. You will get the following result as follows

{

  "status" : 200,

  "name" : "Scarlet Scarab",

  "cluster_name" : "elasticsearch",

  "version" : {

    "number" : "1.6.0",

    "build_hash" : "cdd3ac4dde4f69524ec0a14de3828cb95bbb86d0",

    "build_timestamp" : "2015-06-09T13:36:34Z",

    "build_snapshot" : false,

    "lucene_version" : "4.10.4"

     },

         "tagline" : "You Know, for Search"

         }

14. Now open visual studio and open a console application.
15. Install NEST using package manager console using following code

PM> Install-Package NEST

            

     
      Now your application has a reference to Elasticsearch.

Let’s jump into coding stuff.

Class:

public class SearchBook

    {

         public int BookId { get; set; }

         public string BookTitle { get; set; }

         public string BookAuthor { get; set; }

    }

First add Namespaces required:

using Nest;

Code:

       var node = new Uri("http://localhost:9200");

     //” http://localhost:9200”---server where Elasticsearch instance is hosted

   var settings = new ConnectionSettings( node, defaultIndex: "my-application");

  // defaultIndex- Is by default and we can change it to our specification later while creating index.

      var client = new ElasticClient(settings);

Above code is common while creating, updating and deleting the index. I will explain one by one.

 Create Index: Let’s create an index for an object.

           SearchBook objSearchBook = new SearchBook();

         //Load the class object, you can load your object with data from sql or any other db

           objSearchBook.BookId = 1;

           objSearchBook.BookTitle = "Fantastic";

           objSearchBook.BookAuthor = "Sam";

       // Create Index -- Index is generated with book id as a primary key

         var CreateIndexResponse= client.Index(objSearchBook,i=>     i.Index("index").Type("search").Id(objSearchBook.BookId));

NOTE: Before running this code make sure that the “ElasticSearch” service is running, which I did it using command prompt (Don’t close the prompt).

      After executing last line you will get the response in “CreateIndexResponse”. And the result is as shown below:

In the response, you can check the “Created” value is “True”, it means the index generated successfully. Huh! You did it….

  Search: Let’s search the book details based on id and author name.

var searchResults = client.Search<SearchBook>(s => s.Index("index").Type("search").From(0).Size(10).Query(q => q.Term(p => p.BookId, 1)));

NOTE: Before running this code make sure that the “ElasticSearch” service is running, which I did it using command prompt (Don’t close the prompt).

After executing the above line of code, you can see the result in “searchResults” as shown below image:

Search Based On Book ID:

string searchString = "sam";

var SearchBkByBookId = client.Search<SearchBook>(s => s.Index("index").Type("search").From(0).Size(10).Query(q => q.Match(m => m.OnField(p => p. BookAuthor).Query(searchString))));

In the above query “searchString” is matched with only BookAuthor column values, if you need to match the “searchString” with multiple columns, you can use “OR” condition within “Match” query.

Note:

We can use “Term” or “Match” query, the problem with the “Term” query which I used in the first search method is that query does not analyze the search term. That’s why “Term” query are case sensitive.

The “Match” query uses the same analyzer which is used while generating the index. Hence “Match” queries are not case sensitive. So you use the proper query based on your search specifications.

“From()” and “Size()” is to limit your search result. It’s like start and end point. Consider if we got 500 hits after searching for a text, and we need only top 50 results the just mention “From(0)” and “Size(50)”.

There many other ways to implement search, you can find them here.

Bulk Indexing:

When we have more records to index, we can use Bulk() API. 

For trial purpose I am adding two objects to a class list. I hope you will get data from data source to the list.

 SearchBook objSearchBook = new SearchBook();

 List<SearchBook> lstSearchBook = new List<SearchBook>();

  //Load the class object, you can load your object with data from sql or any other db

            objSearchBook.BookId = 1;

            objSearchBook.BookTitle = "Fantastic";

            objSearchBook.BookAuthor = "Sam";

            lstSearchBook.Add(objSearchBook);

            objSearchBook = new SearchBook();

            objSearchBook.BookId = 2;

            objSearchBook.BookTitle = "Great";

            objSearchBook.BookAuthor = "Sunny";

            lstSearchBook.Add(objSearchBook);

// Bulk Index Generation-- We will index the lstSearchBook list objects.

            var descriptor = new BulkDescriptor();

            foreach (SearchBook book in lstSearchBook)

            {

                descriptor.Index<SearchBook>(op => op.Index("index").Type("searchbooks").Document(book));

            }

            var BulkIndex = client.Bulk(descriptor);         

After executing the last line, you will get the response in "BulkIndex" as shown below:

I hope my post helps you to understand and implementing Elasticsearch in your application.