Building Elasticsearch from source on macOS

The Elasticsearch Enrich API has, at last, been merged into master! It will be part of 7.5 fairly soon. 🎉

This is fantastic news as it’s a feature I’ve been looking forward to for some time. I’ll explain why that’s the case in a later post but I wanted to get to grips with how the new API works right now, and to do that, I had to build Elasticsearch from source.

Building the project is a simple task but there’s a bit of groundwork to do before the build will proceed. I use Homebrew for installing most dependencies and I’ll assume you have that installed already.

Java 11

You will need to have Java 11 installed for the build to run. If you don’t have it installed already, and don’t want to register with Oracle to download the official JDK, there’s a good thread here that describes the process. All you need is:

brew update
brew tap homebrew/cask-versions
brew install java11

Java 12 JDK

It seems that the Elasticsearch is in a JAR Hell-like situation. The build also requires the Java 12 JDK to run. Building with Java 11 will simply tell you that you need a Java 12 JDK as well.

As Java 12 is end-of-life, you’d expect Java 13 to be a suitable replacement. Some of the Elasticsearch dependencies, however, will fail to build with Java 13 (Gradle and, I think, Groovy are to ‘blame’) and the team are waiting for updates for Java 13 to be supported.

For now, you’ll need to grab it from the archive, extract the download and put it into /Library/Java/JavaVirtualMachines.

Gradle

Gradle needs to be installed but Homebrew can take care of that for us:

brew install gradle

Cloning the repository

Clone the Elasticsearch repository using git:

git clone https://github.com/elastic/elasticsearch.git

This will bring down the master branch, which is the bleeding edge.

Start the build

You’ll first need to set an environment variable to point to the Java 12 JDK, then kick off Gradle:

export JAVA_HOME="/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/"
./gradlew assemble

The build took almost 20 minutes on my 2017 MacBook Pro.

Run the very latest

The builds will go into the distribution directory. Find the one you need, copy it to a different location, and run it as you normally would.

In my case, I copied the archive in distributions/archives/no-jdk-darwin-tar/build/distributions/ directory into my home directory, expanded it, set a new JAVA_HOME (back to Java 11), did basic Elasticsearch config in the YML file, and ran the binary. Hitting localhost:9200 gave this:

{
  "name" : "node-1",
  "cluster_name" : "master-snapshot",
  "cluster_uuid" : "RfU9RRA3T2GnVK3yFB396Q",
  "version" : {
    "number" : "8.0.0-SNAPSHOT",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "df83eb983c5f5cdc50c3d2c3d080b4c56bc571f4",
    "build_date" : "2019-10-23T20:11:54.037351Z",
    "build_snapshot" : true,
    "lucene_version" : "8.3.0",
    "minimum_wire_compatibility_version" : "7.6.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

Everything looks good and I can start experimenting with the Enrich API.

Best of luck if you decide to give this a go!