Subversion Repositories Integrator Subversion

================================================================================

  Licensed to the Apache Software Foundation (ASF) under one or more

  contributor license agreements.  See the NOTICE file distributed with

  this work for additional information regarding copyright ownership.

  The ASF licenses this file to You under the Apache License, Version 2.0

  (the "License"); you may not use this file except in compliance with

  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,

  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

  See the License for the specific language governing permissions and

  limitations under the License.

================================================================================

            ====================================================

            Building The Apache Tomcat 9.0 Servlet/JSP Container

            ====================================================

This project contains the source code for Tomcat 9.0, a container that

implements the Servlet 4.0, JSP 2.3, EL 3.0, WebSocket 1.1 and JASPIC 1.1

specifications from the Java Community Process <https://www.jcp.org/>.

Note: If you just need to run Apache Tomcat, it is not necessary to build

it. You may simply download a binary distribution. It is cross-platform.

Read RUNNING.txt for the instruction on how to run it.

In order to build a binary distribution version of Apache Tomcat from a

source distribution, do the following:

(1) Download and Install a Java Development Kit

 1. If the JDK is already installed, skip to (2).

 2. Download a version 17 or later of Java Development Kit (JDK) release (use

    the latest update available for your chosen version) from one of:

        http://www.oracle.com/technetwork/java/javase/downloads/index.html

        http://openjdk.java.net/install/index.html

        or another JDK vendor.

    Note regarding later versions of Java:

      As documented elsewhere, one of components in Apache Tomcat includes

      a private copy of the Apache Commons DBCP 2 library.

      The JDBC interfaces implemented by DBCP frequently change in non-backwards

      compatible ways between versions of the Java SE specification. Therefore,

      it is likely that DBCP 2 will only compile with the specific version of Java

      listed above and that compilation will fail if a later version of Java is

      used.

      See Apache Commons DBCP 2 project web site for more details on

      available versions of the library and its requirements,

        https://commons.apache.org/dbcp/

 3. Install the JDK according to the instructions included with the release.

 4. Set an environment variable JAVA_HOME to the pathname of the directory

    into which you installed the JDK release.

(2) Install Apache Ant version 1.10.2 or later on your computer.

 1. If Apache Ant version 1.10.2 or later is already installed on your

    computer, skip to (3).

 2. Download a binary distribution of Ant from:

        https://ant.apache.org/bindownload.cgi

 3. Unpack the binary distribution into a convenient location so that the

    Ant release resides in its own directory (conventionally named

    "apache-ant-[version]").

    For the purposes of the remainder of this document, the symbolic name

    "${ant.home}" is used to refer to the full pathname of the release

    directory.

 4. Create an ANT_HOME environment variable to point the directory

    ${ant.home}.

 5. Modify the PATH environment variable to include the directory

    ${ant.home}/bin in its list.  This makes the "ant" command line script

    available, which will be used to actually perform the build.

(3) Building Tomcat 9.0

(3.1) Checkout or obtain the source code for Tomcat 9.0

Clone the source using git, then checkout a specific major branch or

main for the latest code development, or download and unpack a source

package.

 *  Tomcat GitHub repository URL:

        https://github.com/apache/tomcat

 *  Source packages can be downloaded from:

        https://tomcat.apache.org/download-90.cgi

The location where the source has been placed will be further referred as

${tomcat.source}.

The Tomcat local build process does not modify line-endings. The svn repository

is configured so that all files will be checked out with the line-ending

appropriate for the current platform. When using a source package you should

ensure that you use the source package that has the appropriate line-ending

for your platform:

  zip    -> CRLF

  tar.gz -> LF

Note that the release build process does modify line-endings to ensure that

each release package has the appropriate line-endings.

(3.2) Building

 1. The build is controlled by creating a ${tomcat.source}/build.properties

    file.

    It is recommended to always create the file, because of unfortunate

    default value of base.path property. You may start with the following

    content for the file:

        # ----- Default Base Path for Dependent Packages -----

        # Replace this path with the directory path where dependencies binaries

        # should be downloaded

        base.path=/home/me/some-place-to-download-to

 2. Configure base.path property by adding it to the

    ${tomcat.source}/build.properties file.

    The base.path property specifies the place where Tomcat dependencies

    required by the build are downloaded. It is recommended to place this

    directory outside of the source tree, so that you do not waste your

    time re-downloading the libraries.

* NOTE: The default value of the base.path property configures the build script

  to download the libraries required to build Tomcat to the

  ${user.home}/tomcat-build-libs directory.

* NOTE: Users accessing the Internet through a proxy must use the properties

  file to indicate to Ant the proxy configuration.

  The following properties should be added to the ${tomcat.source}/build.properties

  file.

        proxy.use=true

        proxy.host=proxy.domain

        proxy.port=8080

        proxy.user=username

        proxy.password=password

  See Apache Ant documentation for the <setproxy> task for details.

 3. Go to the sources directory and run Ant:

        cd ${tomcat.source}

        ant

    This will execute the "deploy" target in build.xml.

    Once the build has completed successfully, a usable Tomcat installation

    will have been produced in the ${tomcat.source}/output/build directory,

    and can be started and stopped with the usual scripts.

    Note that the build includes Tomcat documentation, which can be found

    in the output/build/webapps/docs directory.

    The path of the output directory can be controlled by specifying the

    "tomcat.output" property in the build.properties file.

* NOTE: Do not run the build as the root user. Building and running Tomcat

  does not require root privileges.

(4) Updating sources and rebuilding

It is recommended that you regularly update the downloaded Tomcat 9.0

sources using your git client.

For a quick rebuild of only modified code you can use:

    cd ${tomcat.source}

    ant

(5) Special builds

There are several targets in Tomcat build files that are useful to be

called separately. They build components that you may want to build

quickly, or ones that are included in the full release and are not built

during the default "deploy" build.

(5.1) Building documentation

The documentation web application is built during the default "deploy"

build.

It can be built quickly by using the following commands:

    cd ${tomcat.source}

    ant build-docs

The output of this command will be found in the following directory:

    output/build/webapps/docs

The API documentation (Javadoc) is built during a "release" build. It is

easy to build it separately by using the following commands:

    cd ${tomcat.source}

    ant javadoc

The output of this command will be found in the following directories:

    output/dist/webapps/docs/api

    output/dist/webapps/docs/elapi

    output/dist/webapps/docs/jspapi

    output/dist/webapps/docs/servletapi

(5.2) Building the extras (commons-logging, webservices etc.)

These components are documented on the "Additional Components"

(extras.html) page of documentation. They are built during a "release"

build.

You can build them by using the following commands:

    cd ${tomcat.source}

    ant extras

(5.3) Building the embedded packages

These are built during a "release" build.

You can build them by using the following commands:

    cd ${tomcat.source}

    ant embed

(5.4) Building the Windows installer

The Windows installer uses the NSIS installer framework.

It can be build on Windows, on any other platform which provides

the Wine Windows emulator or the NSIS binary "makensis".

Linux and MacOS are platforms, on which you can install Wine or

"makensis".

Selecting between Wine and makensis on non-Windows platforms can

be done by setting the ant property "nsis.tool" to either "wine"

or "makensis" in build.properties.

If you want to sign the installer, you will need to set some

properties with names prefixed with "codesigning" in your build.properties.

For details see the targets "jsign-installer" and "jsign-uninstaller"

in build.xml and the default property values in build.properties.default.

(6) Building a full release (as provided via the ASF download pages)

 1. Remark on building the Windows installer

    A full release includes the Windows installer which requires a Windows

    environment, an installed Wine emulator or an installed native

    "makensis" binary from the NSIS project. Creating a reproducible

    installer using the "makensis" option needs a special build of "makensis".

    For details see below.

    Provided that Wine or "makensis" is available on non-Windows platforms,

    a full release build may be made on Windows, Linux or MacOS.

    If you do not want to build the Windows installer, the skip.installer

    property may be set to skip the creation of the Windows installer.

 2. Configure GPG, if needed

    If the released artifacts have to be cryptographically signed with a

    PGP signature, like the official ASF releases are, the following

    property can be added to the build.properties file:

        # Location of GPG executable (used only for releases)

        gpg.exec=/path/to/gpg

    You do not need it if you do not plan to sign the release.

    If "gpg.exec" property does not point to an existing file, it will be

    ignored and this feature will be deactivated.

    You will be prompted for the GPG passphrase when the release build

    starts, unless "gpg.passphrase" property is set.

 3. If building the Windows installer on Windows

    If running the build in a UAC enabled environment, building the Windows

    installer requires elevated privileges. The simplest way to do this is to

    open the command prompt used for the build with the "Run as administrator"

    option.

 4. Configure the code signing service

    ASF committers performing official releases will need to configure the code

    signing service so that the Windows installer is signed during the build

    process. The following properties need to be added to the build.properties

    file:

        # Location of GPG executable (used only for releases)

        gpg.exec=/path/to/gpg

        # Code signing of Windows installer

        do.codesigning=true

        codesigning.storepass=request-via-pmc

        codesigning.keypass=request-via-pmc

    Release managers will be provided with the necessary credentials by the PMC.

    If you want to verify the installer from a release by rebuilding it,

    you can use the detached signatures provided in the official releases.

    In this case you have to use the sources from the source release

    archive, which contains these signatures. You also have to build

    on Windows, use Wine, or use a special build of makensis to create

    a reproducible installer, that fits the signature files.

 5. Using a special "makensis" build on non-Windows (optional)

    If you want to build a reproducible installer on non-Windows

    by using "makensis" instead of "Wine", you need to build

    "makensis" from a source download yourself. To build "makensis",

    you need python, scons and a C compiler. You can build "makensis"

    via an Ant target:

    ant local-makensis

    or you can build it manually using the following recipe.

    First detect the needed version by looking for nsis.version

    in build.properties.default.

    Download and extract NSIS for Windows by running "ant download-dist". Note

    the NSIS installation path from the output or by checking "nsis.bin.home" in

    build.properties.default. For example by default:

    nsis.bin.home=${base.path}/nsis-${nsis.version}

    The corresponding sources will be downloaded to "nsis.src.home" in

    build.properties.default. For example by default:

    nsis.src.home=${base.path}/nsis-${nsis.version}-src

    Install the scons Python software construction tool.

    Run the following command in the extracted NSIS source directory:

        scons UNICODE=yes PREFIX=${base.path}/nsis-${nsis.version}/Bin SKIPPLUGINS=all SKIPUTILS=all SKIPMISC=all NSIS_CONFIG_CONST_DATA_PATH=no VERSION=${nsis.version} install-compiler

    You need to specify the correct value in VERSION (this ends up in the installer binary

    so it needs to match). The PREFIX is not important, but you need to make sure

    that the created binary "makensis" gets finally put into the "Bin" directory of

    the NSIS binary distribution that "ant download-dist" installed. When using

    "NSIS_CONFIG_CONST_DATA_PATH=no" the binary is independent of the value of "PREFIX".

    In addition set the ant property "nsis.tool" to "makensis" in

    build.properties.

 6. Build the release:

    Apache Tomcat releases are fully reproducible.

    Release managers producing release builds must follow the following

    procedure:

    cd ${tomcat.source}

    ant pre-release

    ant release

    git commit -a -m "Tag <version-number>"

    git tag <vesion-number>

    git push origin <version-number>

    ant release

    git reset --hard HEAD~1

    The output from either 'ant release' call may be uploaded as the official

    release since they will be identical. It is recommended that the output from

    the second call is used.

    Anyone wishing to reproduce an official build must do so from an official

    source release. The build tool chain defined in build.properties.release

    must be used to reproduce an official build. Once unpacked to

    ${tomcat.source}, the following steps must be followed

    cd ${tomcat.source}

    ant release

    Following the same steps without using the defined build tool chain will

    create a release that is functionally the same as an official release but

    not bit for bit identical.

(7) Tests

(7.1) Running Tomcat tests

Tomcat includes a number of junit tests. The tests are not run when a

release is built. There is separate command to run them.

To run the testsuite use the following command:

    cd ${tomcat.source}

    ant test

It is advisable to redirect output of the above command to a file for later

inspection.

The JUnit reports generated by the tests will be written to the following

directory:

    output/build/logs

By default the testsuite is run three times to test 3 different

implementations of Tomcat connectors: NIO, NIO2 and APR. (If you are not

familiar with Tomcat connectors, see config/http.html in documentation for

details).

The 3 runs are activated and deactivated individually by the following

properties, which all are "true" by default:

    execute.test.nio=true

    execute.test.nio2=true

    execute.test.apr=true

The APR connector can be tested only if Tomcat-Native library binaries are

found by the testsuite. The "test.apr.loc" property specifies the directory

where the library binaries are located.

By default the "test.apr.loc" property specifies the following location:

    output/build/bin/

If you are on Windows and want to test the APR connector you can put the

tcnative-1.dll file into ${tomcat.source}/bin/ and it will be copied

into the above directory when the build runs.

The unit tests include tests of the clustering functionality which require

multicast to be enabled. There is a simple application provided in the Tomcat

test source (org.apache.catalina.tribes.TesterMulticast) that can be used to

check if a machine supports multicast. Notes on enabling multicast for different

operating systems are provided in the Javadoc for that class.

(7.2) Running a single test

It is possible to run a single JUnit test class by adding the "test.entry"

property to the build.properties file. The property specifies the name of

the test class.

For example:

    test.entry=org.apache.catalina.util.TestServerInfo

It is possible to further limit such run to a number of selected test

methods by adding "test.entry.methods" property. The property specifies a

comma-separated list of test case methods.

For example:

    test.entry=org.apache.el.lang.TestELArithmetic

    test.entry.methods=testMultiply01,testMultiply02

(7.3) Running a set of tests

It is possible to run a set of JUnit test classes by adding the "test.name"

property to the build.properties file. The property specifies an Ant

includes pattern for the fileset of test class files to run.

The default value is "**/Test*.java", so all test classes are being

executed (with few exceptions - see build.xml for several exclude patterns).

You can include multiple patterns by concatenating them with a comma (",")

as the separator.

For example:

    test.name=**/TestSsl.java,**/TestWebSocketFrameClientSSL.java

You can exclude specific JUnit test classes by adding the "test.exclude"

property to the build.properties file. The property specifies an Ant

excludes pattern for the fileset of test class files to exclude form the run.

The default value is empty, so no classes are excluded. The syntax is the same

as for the property "test.name".

(7.4) Other configuration options

 1. It is possible to configure the directory where JUnit reports are

 written to. It is configured by "test.reports" property. The default

 value is

        output/build/logs

 2. It is possible to enable generation of access log file when the tests

 are run. This is off by default and can be enabled by the following

 property:

        test.accesslog=true

 The "access_log.<date>" file will be written to the same directory as

 JUnit reports,

        output/build/logs

 3. The testsuite respects logging configuration as configured by

 ${tomcat.source}/conf/logging.properties

 The log files will be written to the temporary directory used by the

 tests,

        output/test-tmp/logs

 4. It is possible to configure formatter used by JUnit reports.

 Configuration properties are "junit.formatter.type",

 "junit.formatter.extension" and "junit.formatter.usefile".

 For example the following property deactivates generation of separate report

 files:

        junit.formatter.usefile=false

 5. It is possible to speed up testing by letting JUnit to run several

 tests in parallel.

 This is configured by setting "test.threads" property. The recommended

 value is one thread per core.

 6. Optional support is provided for the Cobertura code coverage tool.

NOTE: Cobertura is licensed under GPL v2 with parts of it being under

      Apache License v1.1. See https://cobertura.github.io/cobertura/ for details.

      Using it during Tomcat build is optional and is off by default.

 Cobertura can be enabled using the following properties:

        test.cobertura=true

        test.threads=1

 Using Cobertura currently requires setting test.threads configuration

 property to the value of 1. Setting that property to a different value

 will deactivate code coverage.

 The report files by default are written to

        output/coverage

 7. The performance tests are written to run on reasonably powerful machines

    (such as a developer may use day to day) assuming no other resource hungry

    processes are running.

    Performance tests may be an absolute test (how long to complete a number

    of iterations of an operation or set of operations) or may be a relative

    test (how long two or more different approaches take to generate the same

    result). The absolute tests may be destructive in that they run until the

    system runs out of resources.

    Where there is no benefit in running an absolute performance test as part

    of a standard test run, the test will be excluded by naming it

    Tester*Performance.java.

    The relative tests are included as part of a standard test run however,

    where the assumptions made about host capabilities are  not true (e.g. on

    CI systems running in virtual machine) the tests may be deactivated by

    using the following property:

        test.excludePerformance=true

 8. Some tests are require large heaps (e.g. 8GB). The CI systems used by the

    project either cannot support heaps of this size or do not support them by

    default. These tests are therefore disabled by default and may be enabled by

    using the following property:

        test.includeLargeHeap=true

 9. Some tests include checks that the access log valve entries are as expected.

    These checks include timings. On slower / loaded systems these checks will

    often fail. The checks may be relaxed by using the following property:

        test.relaxTiming=true

 10. It is known that some platforms (e.g. OSX El Capitan) require IPv4 to

     be the default for the multicast tests to work. This is configured by

     the following property:

        java.net.preferIPv4Stack=true

 11. By default the output of unit tests is sent to the console and can be

     quite verbose. The output can be deactivated by setting the property:

        test.verbose=false

(8) Source code checks

(8.1) Checkstyle

NOTE: Checkstyle is licensed under LGPL. Using Checkstyle during Tomcat

      build is optional and is off by default.

      See http://checkstyle.sourceforge.net/ for more information.

Tomcat comes with a Checkstyle configuration that tests its source code

for certain conventions, like presence of the license header.

To enable Checkstyle, add the following property to build.properties file:

    execute.validate=true

Once Checkstyle is enabled, the check will be performed automatically

during the build. The check is run before compilation of the source code.

To speed-up repeated runs of this check, a cache is configured. The cache

is located in the following directory:

    output/res/checkstyle

It is possible to run the check separately by calling the "validate"

target. The command is:

    cd ${tomcat.source}

    ant -Dexecute.validate=true validate

(8.2) SpotBugs

NOTE: SpotBugs is licensed under LGPL. Using SpotBugs during Tomcat build is

      optional and is off by default.

      See https://spotbugs.github.io/ for more information.

To enable SpotBugs, add the following property to build.properties file:

    execute.spotbugs=true

To compile Tomcat classes and generate a SpotBugs report, call the

"spotbugs" target. For example:

    cd ${tomcat.source}

    ant -Dexecute.spotbugs=true spotbugs

The report file by default is written to

    output/spotbugs

(8.3) End-of-line conventions check

You usually would not need to run this check. You can skip this section.

Apache Tomcat project has convention that all of its textual source files,

stored in the Git repository, use Unix style LF line endings.

This test is used by developers to check that the source code adheres to

this convention. It verifies that the ends of lines in textual files are

appropriate. The idea is to run this check regularly and notify developers

when an inconsistency is detected.

The command to run this test is:

    cd ${tomcat.source}

    ant validate-eoln