omniosce Logo OmniOS Community Edition is a free and open-source Unix operating system derived from OpenSolaris and based on Illumos. Developers forked OpenSolaris after Oracle Corporation discontinued it, in order to continue development and distribution of the source code. The OmniOSCE project is stewarded by the OmniOS Community Edition Association. OmniOSCEs goal is to produce a self-hosting, minimalist Illumos-based release suitable for production deployment.

The Image Packaging System, also known as IPS, is a cross-platform (written in Python) package management system created by the OpenSolaris community in coordination with Sun Microsystems. IPS is designed to eliminate some long-standing issues with previous software distribution, installation, and maintenance mechanisms that have caused significant problems for Solaris users, developers, and maintainers.

Today I will compile and build an application from source to publish to a local IPS Repository. I have chosen to create the Apache httpd Server as it demonstrates dealing with dependencies & using SMF services. That will be as far as I get with today’s post and it will be followed up in a new post, by creating the Apache httpd package and publishing it to an IPS repository.

Building Apache Under OmniOSCE

Here I outline the build of Apache httpd Server. I build this into /opt/pbdigital. For the system to find the binaries, libraries & manual pages, I have added the path /opt/pbdigital/bin & /opt/pbdigital/sbin to the system wide environment variables declared in /etc/defaults/login.

The following path declarations should be as follows:

PATH=/usr/bin:/usr/sbin:/sbin:/opt/ooce/bin:/usr/gnu/bin:/opt/pbdigital/bin
SUPATH=/usr/sbin:/sbin:/opt/ooce/sbin:/usr/bin:/opt/ooce/bin:/opt/pbdigital/bin:/opt/pbdigital/sbin

Installing Development Software to Build Apache

First we install the GCC compiler and Gnu Make so we can build Apache with these tools:

root@omnios:~# pkg install gcc8 gnu-make
           Packages to install:  7
           Mediators to change:  1
            Services to change:  1
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                                7/7     3114/3114    71.2/71.2  325k/s

PHASE                                          ITEMS
Installing new actions                     3887/3887
Updating package state database                 Done 
Updating package cache                           0/0 
Updating image state                            Done 
Creating fast lookup database                   Done 
Reading search index                            Done 
Updating search index                            7/7 
Updating package cache                           2/2 

Starting the Build for Apache Portable Runtime

First I create the directories where I will build Apache and its dependencies.

# mkdir -p ips-builds/{apr,apr-util,httpd}

The first dependency we need to build is Apache Runtime. The source tarballs can be fetched, and the program can be built as follows:

# cd ips-builds/apr
# wget https://apache.brunneis.com//apr/apr-1.7.0.tar.gz
# tar xvf apr-1.7.0.tar.gz
# cd apr-1.7.0
# CFLAGS="-g -O -m64" \
>   ./configure \
>   --build=x86_64-pc-solaris2.11 \
>   --prefix=/opt/pbdigital/apr-1.7.0 \
>   --enable-layout=opt

CFLAGS: CFLAGS are either the name of environment variables or of Makefile variables that can be set to specify additional switches to be passed to a compiler in the process of building. The following variables have been set:

  • “-g”: Produce debugging information in the operating system’s native format (stabs, COFF, XCOFF, or DWARF). GDB can work with this debugging information.,

  • “-O”: turns on the optimization flags. See GCC manpage for details.

  • “-m64”: Generate code for a 64-bit environment.

See the GCC manpage for more details.

–build: is the architecture of the build machine. This had to be declared so as to build both static and dynamic libraries.

–prefix: By default, make install will install all the files in /usr/local/bin, /usr/local/lib etc. You can specify an installation prefix other than /usr/local using –prefix, for instance –prefix=$HOME.

–enable-layout: Configure the source code and build scripts to assume an installation tree based on the layout LAYOUT. This allows you to separately specify the locations for each type of file within the Apache Runtime installation. The config.layout file contains several example configurations, I have used “opt” which is suitable for the omniOS packaging guidelines.

NOTE: After reviewing the output of ./configure, & you are concerned about the “rm: libtoolT: No such file or directory” message, you can see here for an explanation.

NOTE: During the configure stage I also see that SCTP will not be supported in the build.

configure: WARNING: netinet/sctp.h: present but cannot be compiled

I looked high and low for a solution to this, however in the end I feel SCTP support will only be needed in very few use cases, so have not followed up on it. You can look here to see the reason of SCTP failing.

Build & Install for Apache Runtime

I have chosen to use gmake, though these 3 Apache source tarballs will work under make.

When running gmake, you will be presented with the following warnings:

locks/unix/proc_mutex.c: In function 'proc_mutex_pthread_cleanup':
locks/unix/proc_mutex.c:577:25: warning: passing argument 1 of 'munmap' from incompatible pointer type [-Wincompatible-pointer-types]
     if (munmap(mutex->os.pthread_interproc, sizeof(proc_pthread_mutex_t))) {
In file included from /home/philip/dev/ips-builds/apr/apr-1.7.0/include/arch/unix/apr_arch_proc_mutex.h:61,
                 from locks/unix/proc_mutex.c:19:
/usr/include/sys/mman.h:245:19: note: expected 'caddr_t' {aka 'char *'} but argument is of type 'pthread_mutex_t *' {aka 'struct _pthread_mutex *'}
 extern int munmap(caddr_t, size_t);
                   ^~~~~~~
locks/unix/proc_mutex.c: In function 'proc_mutex_pthread_create':
locks/unix/proc_mutex.c:596:37: warning: assignment to 'pthread_mutex_t *' {aka 'struct _pthread_mutex *'} from incompatible pointer type 'caddr_t' {aka 'char *'} [-Wincompatible-pointer-types]
     new_mutex->os.pthread_interproc = mmap(NULL, sizeof(proc_pthread_mutex_t)

This can be cleaned up with the following diff that was originally resolved on the Apache Mailing List.

# diff locks/unix/proc_mutex.c.orig locks/unix/proc_mutex.c      
577c577
<     if (munmap(mutex->os.pthread_interproc, sizeof(proc_pthread_mutex_t))) {
---
>     if (munmap((void*)mutex->os.pthread_interproc, sizeof(proc_pthread_mutex_t))) {
595c595
<     new_mutex->os.pthread_interproc = mmap(NULL, sizeof(proc_pthread_mutex_t),
---
>     new_mutex->os.pthread_interproc = (pthread_mutex_t*)mmap(NULL,sizeof(proc_pthread_mutex_t),

Now you can compile and install without further problems:

# gmake
# gmake install

This will install apr under /opt/pbdigital/. You can check that we have built 64-bit binaries & libraries, with the file command:

# file /opt/pbdigital/apr-1.7.0/lib/libapr-1.so.0.7.0
/opt/pbdigital/apr/lib/libapr-1.so.0.7.0: ELF 64-bit LSB dynamic lib AMD64 Version 1, dynamically linked, not stripped

Starting the Build for Apache Portable Runtime Utilities

First install berkeley-db package needed for apr-util.

pkg install bdb

The second dependency we build is Apache Portable Runtime Utilities. There are many compile time options and I have chosen to mimic the choices as used by the FreeBSD ports. These are openssl, crypto, cdbm & berkeley-db. Again we download the source tarball and compile the program as follows:

# cd ips-builds/apr-util
# wget https://apache.brunneis.com//apr/apr-util-1.6.1.tar.gz
# tar xvf apr-util-1.6.1.tar.gz
# cd apr-util-1.6.1
# CFLAGS="-g -O -m64 -I/opt/ooce/include" \
>   LDFLAGS="-L/opt/ooce/lib/amd64 -R/opt/ooce/lib/amd64" \
>   LIBS="-ldb-5.3" \
>   ./configure \
>   --build=x86_64-pc-solaris2.11 \
>   --prefix=/opt/pbdigital/apr-util-1.6.1 \
>   --enable-layout=opt \
>   --with-apr=/opt/pbdigital/apr-1.7.0 \
>   --with-openssl \
>   --with-crypto \
>   --with-gdbm \
>   --with-berkeley-db=/opt/ooce/include:/opt/ooce/lib/amd64

With the CFLAGS I have added the -I declaration, for gmake to search /opt/ooce/include directory for includes files, which is where the Berkeley-DB include files can be found.

LDFLAGS pass information about libraries to the configure script. The -L declaration is used to point the compiler, gmake, to the location of where the Berkeley-DB libraries are. The -R declaration determines the runtime location for when the newly compiled program is running.

LIBS points the compiler to a dynamic library. In this case libdb-5.3.0.so. The format is -l and the shared object library, minus the prefix of lib & the .so (shared object) extension.

Along with the options explained above, I have altered the –prefix location to install apr-util in its own directory. Also, apr-util needs to know the location of the apr code that we have just built and we can set the location with –with-apr=/opt/pbdigital/apr.

NOTE: No matter how many different times I tried to set the LDFLAGS via the configure script, these LDFLAGS refused to be set in the Makefile. Therefore you will need to edit the Makefile and make sure the line beginning with APRUTIL_LDFLAGS is as follows:

APRUTIL_LDFLAGS = -L/opt/ooce/lib/amd64 -R/opt/ooce/lib/amd64

Also that the line immediately below that starts with APRUTIL_LIBS includes the -ldb-5.3 library.

APRUTIL_LIBS = -lexpat /opt/pbdigital/apr-1.7.0/lib/libapr.la -luuid -lsendfile -lsocket -lnsl -lpthread -ldb-5.3

Build & Install for Apache Portable Runtime Utilities

As above we build and install apr-util with gmake.

Without altering the source file crypto/apr_crypto.c, the compiler will fail when encountering memset_s. The error will be as below:

crypto/apr_crypto.c: In function 'apr_crypto_memzero':
crypto/apr_crypto.c:160:16: warning: implicit declaration of function 'memset_s'; did you mean 'memset'? [-Wimplicit-function-declaration]
         return memset_s(buffer, (rsize_t)size, 0, (rsize_t)size);
                ^~~~~~~~
                memset
crypto/apr_crypto.c:160:34: error: 'rsize_t' undeclared (first use in this function); did you mean 'ssize_t'?
         return memset_s(buffer, (rsize_t)size, 0, (rsize_t)size);
                                  ^~~~~~~
                                  ssize_t
crypto/apr_crypto.c:160:34: note: each undeclared identifier is reported only once for each function it appears in
crypto/apr_crypto.c:160:42: error: expected ')' before 'size'
         return memset_s(buffer, (rsize_t)size, 0, (rsize_t)size);
                                          ^~~~
                                          )
gmake[1]: *** [/home/philip/dev/ips-builds/apr-util/apr-util-1.6.1/build/rules.mk:206: crypto/apr_crypto.lo] Error 1
gmake[1]: Leaving directory '/home/philip/dev/ips-builds/apr-util/apr-util-1.6.1'
gmake: *** [/home/philip/dev/ips-builds/apr-util/apr-util-1.6.1/build/rules.mk:118: all-recursive] Erro

memset_s function was proposed and introduced in C11. More specifically, Annex K of C11. This compile failure can be fixed with allowing C11 - Annex K extensions.

# diff crypto/apr_crypto.c.orig crypto/apr_crypto.c
16a17,18
> #define __STDC_WANT_LIB_EXT1__ 1
> 

Onwards with the compile and install:

# gmake
# gmake install

Starting the Build for Apache httpd

Now with the dependencies resolved, we can start on building the httpd daemon. The long list of options I have included mimic the options of the FreeBSD port. Once more we download the source tarball and compile the program as follows:

# cd ips-builds/httpd
# wget https://ftp.cixug.es/apache//httpd/httpd-2.4.41.tar.gz
# tar xvf httpd-2.4.41.tar.gz
# cd httpd-2.4.41
# CFLAGS="-m64 -I/opt/ooce/bdb-5.3.28/include" \
>   CPPFLAGS="-I/opt/ooce/include" \
>   LDFLAGS="-L/opt/ooce/lib/amd64 -R/opt/ooce/lib/amd64" \
>   LIBS="-ldb-5.3" \
>   ./configure \
>   --build=x86_64-pc-solaris2.11 \
>   --prefix=/opt/pbdigital/httpd-2.4.41 \
>   --enable-layout=opt \
>   --with-apr=/opt/pbdigital/apr-1.7.0 \
>   --with-apr-util=/opt/pbdigital/apr-util-1.6.1 \
>   --enable-authn-dbm \
>   --enable-authn-anon \
>   --enable-authn-dbd \
>   --enable-authn-socache \
>   --enable-authz-dbm \
>   --enable-authz-owner \
>   --enable-authz-dbd \
>   --enable-authnz-fcgi \
>   --enable-auth-form \
>   --enable-auth-digest \
>   --enable-allowmethods \
>   --enable-file-cache \
>   --enable-cache \
>   --enable-cache-disk \
>   --enable-cache-socache \
>   --enable-socache-shmcb \
>   --enable-socache-dbm \
>   --enable-socache-memcache \
>   --enable-watchdog \
>   --enable-macro \
>   --enable-dbd \
>   --enable-dumpio \
>   --enable-buffer \
>   --enable-ratelimit \
>   --enable-ext-filter \
>   --enable-request \
>   --enable-include \
>   --enable-reflector \
>   --enable-substitute \
>   --enable-sed \
>   --enable-deflate \
>   --enable-xml2enc \
>   --enable-proxy-html \
>   --enable-log-debug \
>   --enable-log-forensic \
>   --enable-logio \
>   --enable-mime-magic \
>   --enable-cern-meta \
>   --enable-expires \
>   --enable-ident \
>   --enable-usertrack \
>   --enable-unique-id \
>   --enable-remoteip \
>   --enable-proxy \
>   --enable-proxy-connect \
>   --enable-proxy-ftp \
>   --enable-proxy-http \
>   --enable-proxy-fcgi \
>   --enable-proxy-scgi \
>   --enable-proxy-uwsgi \
>   --enable-proxy-wstunnel \
>   --enable-proxy-ajp \
>   --enable-proxy-balancer \
>   --enable-proxy-express \
>   --enable-proxy-hcheck \
>   --enable-session \
>   --enable-session-cookie \
>   --enable-session-crypto \
>   --enable-session-dbd \
>   --enable-slotmem-shm \
>   --enable-slotmem-plain \
>   --enable-ssl \
>   --enable-dialup \
>   --enable-http2 \
>   --enable-proxy-http2 \
>   --enable-lbmethod-byrequests \
>   --enable-lbmethod-bytraffic \
>   --enable-lbmethod-bybusyness \
>   --enable-lbmethod-heartbeat \
>   --enable-mpms-shared=all \
>   --enable-heartbeat \
>   --enable-heartmonitor \
>   --enable-dav \
>   --enable-asis \
>   --enable-info \
>   --enable-cgid \
>   --enable-cgi \
>   --enable-dav-fs \
>   --enable-dav-lock \
>   --enable-vhost-alias \
>   --enable-negotiation \
>   --enable-imagemap \
>   --enable-actions \
>   --enable-speling \
>   --enable-userdir \
>   --enable-rewrite    

I have altered the –prefix location to install apache-httpd in its own directory. Also, httpd needs to know the location of the apr & apr-util code that we have just built and we can set the location with –with-apr=/opt/pbdigital/apr-1.7.0 & –with-apr-util=/opt/pbdigital/apr-util-1.6.1.

Build & Install for Apache httpd

As above we build and install httpd with gmake.

# gmake
# gmake install

If we search under /opt/pbdigital/httpd-2.4.41 we can see we have an installed version of Apache. We will also have files installed under /etc/opt/pbdigital/httpd-2.4.41 & /var/opt/pbdigital/httpd-2.4.41. We can start the server with /opt/pbdigital/httpd-2.4.41/sbin/apachectl start.

This leads onto the next post I will write, which creates an IPS Package that will cleanly install Apache in an organized manner, including SMF files and also setting the user for the Apache httpd daemon to run under the default web daemon user, webservd. We will create an IPS Repository and publish this IPS package on this IPS Repository Server.

Wrapping Up

That is all for today. If you would like to peek ahead and see what is required to build and publish an IPS package you can take a look here at the Oracle Documentation.