Compiling in MacOS X

From OSUPDOCS
Jump to navigation Jump to search

This page explains several methods to compile code engines using MacOS X.

Using XCode

Most development of NairnMPM and NairnFEA is done in MacOS X and thus compiling on MacOS X is easy. The preferred method is to use XCode (but first you must install Xcode, install command line tools, and get a new compiler). A complete XCode project is located at 

nairn-mpm-fea/Common/Projects/NairnMPM.xcodeproj

This project is called NairnMPM, but it includes all MPM and FEA source code in two targets named NairnMPM and NairnFEA. Once xerces is installed, open the NairnMPM.xcodeproj to compile the code by:

  1. Select the target to compile - either NairnMPM or NairnFEA
  2. Choose build and they will compile and be saved in XCode's derived data folder.

If a linking error occurs, you might have conflicting architectures between the XCode settings and the xerces library you installed. To fix this problem edit both the project and target settings and under the "Architectures" section, set the "Architectures" option to match the architecture you used when installing xerces or recompile xerces if needed.

Compiling Xerces on MacOS X

Before you can compile and run the project, however, you will need an installed version of the xerces library and a copy of the xerces header files. These compiling instructions assume they are installed in the default locations for MacOS X or at: 

/usr/local/lib/libxerces-c.dylib

for the library and at 

/usr/local/include

for the header files. These can be changed if needed by editing the project and target settings. The nairn-mpm-fea project needs access to xerces 3.0 or newer. This libraries can be obtained by downloading the xerces source code from the Apache Software Foundation web site and then building and installing xerces with the following steps:

  1. First install XCode and command line tools.
  2. Open Terminal app and navigate to the xerces folder expanded from the downloaded file.
  3. Configure the code with the command: 
    ./configure CFLAGS="-arch x86_64 -arch arm64" CXXFLAGS="-arch x86_64 -arch arm64"
    where the provided arch options (or architectures) are for Intel chips (x86_64) or the new Silcon M1 chips (arm64); you can pick one for your computer or include both to have a "fat" binary with both architectures. To determine your chips, choose "About This Mac" from the Apple menu.
  4. When the configuration is done, use the following commands: 
    cd src
make
sudo make install
    These commands make the library (but not the unneeded xerces examples). The final install command after make is done, installs both the library and the header files at the default locations listed above. It requires sudo for you to provide your administrator password needed to authenticate installation and the default location (which is /usr/local).
  5. When working with nairn-mpm-fea on a new Mac, the xerces installation only needs to be done once. The only reason to repeat it is when a new xerces version is available and/or the project requires a new version for compatibility.

MacOS X Command Line Compiling

It also possible to compile on MacOS X using a command line approach (after installing XCode and command line tools and after installing xerces and its header files as explained above making sure they are in the specified default locations). You can compile NairnMPM using: 

cd nairn-mpm-fea/NairnMPM/build
make -j 8 SYSTEM=mac-clang

and compile NairnFEA using: 

cd nairn-mpm-fea/NairnFEA/build
make -j 8 SYSTEM=mac-clang

The -j 8 option compiles in parallel where the number is number of available cores on your computer. All source code will be compiled and the executables will be installed in nairn-mpm-fea/NairnMPM/input or nairn-mpm-fea/NairnFEA/input, respectively. You can use an additional make install command to copy each compiled executable to your ~/bin folder if desired.

The above commands assume you have installed the clang-mp compiler. If you prefer to install and use gcc change the SYSTEM=mac-clang lines to SYSTEM=mac instead.

If the command-line compile does not work, the most likely explanation is a problem with the xerces installation. You either have to install it as specified above or edit the makefile to recognize your custom installation. The process is documented in the makefile and involves editing the xercesSo and headersPath variables for your different settings.

You can pass additional parameters to the make command to alter the compilation process (e.g., a CC option can pick a compiler installed on your computer). See comments in the makefile for all the latest options.

Installing XCode

All compiling on Mac requires that you install Xcode (even if you do not plan to use it for anything else). You can get Xcode from the Mac App Store or from the Apple Developer site. It is a large install. Once installed, you then need to install command line tools.

Unfortunately, the compilers provided in Apple's "Command Line Tools" do not support OpenMP used to make these code engines parallel. The two supported solutions are to compile without OpenMP (and lose all advantages of multiprocessor execution) or to install and use the clang-mp compiler. Some notes below explain how to install and use an unsupported GCC compiler.

The current check out is configured to compile in XCode using clang-mp-7.0. Prior check outs are set up to use earlier versions of clang-mp or various version of GCC. It is be easy to convert any checkout to use the compiler you prefer by making a few changes in the XCode project build settings. Alternatively, you can use command line compilation and specify settings for currently installed compiler

Whenever you update your MacOS or Xcode, you may need to repeat some installation steps before you can return to compiling the code engines.

The following sections have more details on installations needed for compiling the code engines.

Installing Command Line Tools

Apple used to install command line tools by default, but stopped doing that in MacOS Lion. You now have to manually install the tools before you can compile code. These steps should work:

  1. After installing XCode on a recent MacOS, launch the app and agree to all options. Once that is done, install command line tools by opening Terminal app and enter: xcode-select --install (but they may already be installed). If this step does not work, you can see MacPorts for possible new instructions. You can also get the command line tools from the Apple developer website and search for command line tools (being sure to get the correct tools for you current Xcode and MacOS versions). You may need to be signed up as an Apple developer for this method.
  2. You may need to agree to XCode license by using Terminal app to enter: sudo xcodebuild -license
  3. Most tools should now be available. You should be able to make and install the xerces library.
  4. To compile the code engines for parallel code, you will additionally need to install a compiler that works with OpenMP (none is provided in the command line tools).

With each new system, Apple seems to creating road blocks to keep you from using you computer for interesting programming. For now, by using a few tricks it is still possible to get around their road blocks. Hopefully these tricks will continue to work in the future.

One issue in the XCode project is that it might not find your xerces library, even if it is in the standard location. If it is not found, the library may be in red in the "External Frameworks and Libraries" folder (or may not). A potential solution is to delete the reference to the library and then add it back. The problem is that you cannot navigate to the /usr/local/lib folder anymore (another new Apple "feature"). Here is a trick to get there:

  1. In the Finder, use the "Go to Folder..." menu command and enter "/usr/local".
  2. After it opens, choose the "Add To Sidebar" menu command. This folder will now appear in all file selection boxes.
  3. Go back to XCode and use command to add files for adding the xerces library.
  4. Go through your new "local" folder and select the xerces library file in /usr/local/lib.

Install Clang-mp

Although Apple using clang for compiling in Xcode, for reasons that are hard to justify, they opted to disable compiling with OpenMP (using flag -fopenmp). The further did not include libomp.dylib, which is needed to run software written for OpenMP. Because doing parallel computations is essential for large calculations, some method to use OpenMP is needed. Two options are:

  1. Hack Apple's installed clang compiler to use -fopenmp and then manually install libomp.dylib. One posted method almost works. It is uncertain if it will continue work and I cannot get it to work in MacOS Sonoma.
  2. Install a separate clang-mp compiler and use that instead of Apple's installed clang compiler. This option is described here.

A good way is to install clang-mp is to use MacPorts. Once the MacPorts tool is installed (and possibly updated for current MacOS), the installation of clang-mp is (usually) easy. All you need is to open the Terminal app and enter: 

sudo port selfupdate
sudo port install clang-11 ld64 +ld64_xcode

The first command above is only needed to make sure your MacPorts is up to date. The second step will install the compiler at 

/opt/local/bin/clang-mp-11

If that binary is not there, a problem occurred. I never experiences a problem until trying in MacOS Sonoma (and maybe newer). the two new issues are:

  1. A MacPorts install command reports that clang-mp versions 11, 12, and 13 are "known to fail." It appears these version can still be installed, but the sudo port install may not finish. If it exits without a binary, repeat the install command until it (hopefully) finishes. Each repeat will pick up where it left off. If it hangs up while installing, force exit with Control-C and repeat again. If it eventually finishes, the install seems to work. I need three repeats for one successful install of clang-mp-11
  1. Installing clang-mp versions 14 or new to not say "known to fail", but the first install command exits with a vague "permission denied" error. Repeating the install command then picks up where it left off and finishes the install. Although these version get installed, a new issue appears when compiling the code. See [#XCode Project Settings|XCode project settings]] for details on changing the Xcode project file to switch new versions of clang-mp.

The downloaded project will be set to use a specific version of clang-mp. To find out which one, open the project in Xcode, find the "Build Settings" for the NairnMPM projects, and scroll the to bottom to see "User-defined" settings. The CC setting in the "User-defined" settings specifies the clang-mp version being used. You can install and use that version or install and use some other version (so far any version works). If you pick a different version, you will have to update the project two ways:

  1. To compile using Xcode, edit the Xcode project settings to use the new version.
  2. To use clang-mp for command line compiling, update the makefile to use your new compiler (see comments in makefile for details).

You can install multiple version of clang-mp, which can be useful if looking at older version of this software. more likely, you will only need on version. If you settle on one version, you can remove older ones with 

sudo port uninstall --follow-dependents clang-xx

where xx is the version you want to uninstall.

XCode Build Settings

Compiling using a clang-mp compiler in Xcode requires some specific project build settings. Various versions of the checkout may have settings for one specific compiler, but you should be able to easily switch to your preferred settings using the details below:

  1. Using clang-mp
    • Compiler for C/C++/Objective-C: select the default compiler.
    • Use menu command Editor→Add Build Setting→Add User-Defined Setting and add setting for CC with value /opt/local/bin/clang-mp-17 (using clang version number you installed).
    • Use menu command Editor→Add Build Setting→Add User-Defined Setting and add setting for LDPLUSPLUS with value /opt/local/bin/clang++-mp-17 (using clang version number you installed).
    • Other C Flags: add -fopenmp.
    • Other C++ Flags: add $(OTHER_CFLAGS) to have same flags used for C++ compile as well.
    • Other Linker Flags: add -fopenmp.
    • Enable Modules (C and Objective-C): set to No
    • Header Search Paths: add /opt/local/include/libomp
  2. Compiling Without OpenMP
    • Compiler for C/C++/Objective-C: select the default compiler.
    • Other C Flags: remove -fopenmp
    • Other C++ Flags: remove -fopenmp
    • Other Linker Flags: none.
    • Comment out #define USE_OPENMP in the MPMPrefix.hpp and FEAPrefix.hpp files.

Note that compiling without OpenMP will create serial code. Because it will be much slower than parallel code, that option is only recommended when no other option works (for some reason).

Possible Changes Needed After Updating MacOS or XCode

Unfortunately, Apple does not do a very good job of preserving your prior settings after a major upgrade and you will often need a complete reinstall of anything installed using MacPorts. Some things that might be needed with each MacOS upgrade are:

  1. Get the latest version of XCode (from developer web site or app store).
  2. Reinstall command line tools in Terminal app by entering xcode-select --install
  3. Agree to Xcode license in Terminal app using sudo xcodebuild -license (and provide your admin password).
  4. You may need to uninstall all installed MacPorts, reinstall MacPorts, and then reinstall all previously installed options. The process is explained here Migrating a MacPorts Installation. Also see above section about migrating when only using ports for this software. Sometimes a "clean" install is best, especially after a MacOS update. See previous links with a brief outline being:
    1. Uninstall all of MacPorts
    2. Install MacPorts for the new MacOS
    3. Install all ports needed for this software (clang-mp above and maybe subversion too). If you have install ports for other uses, you might want to use one of the other approaches on the MacPorts web site when migrating new a new MacOS instead.
  5. To compile in XCode, you may need more changes (but hard to predict what Apple will require next).

Compiling Old Version of the Code

If you need to compile old versions of the code, the XCode project may not work well (because of changes in XCode versions, build settings, or compiler you have installed). One option to compile old version (such as when searching for bugs) is to compile using command line and your currently installed compiler. For example, if working with current code on Mac and the clang-mp compiler, you may be able to compile old versions using the build folders and the command 

make CC=/opt/local/bin/clang-mp-7.0 xercesSo=/usr/local/lib/libxerces-c.dylib
        CFLAGS="-c -O3 -fopenmp -std=c++11 -I/opt/local/include/libomp"
        LFLAGS="-fopenmp -lc++"

The above uses several lines, but it all should be on a single command line.

Deprecated Compiler Options

Use of the GCC compiler is no longer recommended when working with XCode. It may not even work in XCode 8 or newer (at least by methods described below). The GCC compiler does work fine, however, if using only command line methods. The following sections may provide useful information for installing it for that use.

Install GCC 4.8 or Newer

GCC 4.9 or new supports OpenMP and may have some uses (GCC 4.8 and older should be avoided). If you want to use it, a good way to install GCC 4.9 or newer is to use MacPorts. Once MacPorts tool is installed, the installation is explained in this blog. In brief, all you need is to open the Terminal app and enter: 

sudo port selfupdate
sudo port install gcc49

The first command above is only needed to make sure your MacPorts is up to date. The second step will install the compiler at 

/opt/local/bin/g++-mp-4.9

You can change to different version number if needed or when available. If you want to remove an older version of GCC after updating, such as removing GCC 4.8, you should be able to use 

sudo port uninstall --follow-dependents gcc48

To use GCC in XCode, you will need plug in and some specific build settings. For command line compiling, you may need to update the makefile to use you new compiler (see comments in makefile for details). Alternatively, you can use the CC option to specify path to any compiler, such as: 

make SYSTEM=mac CC=/opt/local/bin/g++-mp-4.9

Using GCC 4.9 or Newer in XCode

Using GCC requires a hack. Fortunately, a friend of this project (Hammd Mazhar) has provided a solution. The process is explained on this blog (for XCode 4.x) with an update for Xcode 6.x and newer (so far). Warning: Although the hack still allows one to compile in XCode 8.x, it no long appears possible to link the code using GCC. One alternative is to switch to switch to using clang-mp.

In brief, the goal is to install a custom plug in. The shortest approach is download a plug in and install it:

  1. For GCC 4.8 or GCC 4.9 installed as explained above, download this GCC 4.8 XCode plug in or this GCC 4.9 XCode plug in. For GCC newer then 4.9, you might find a plug in in the repository xcode-gcc.
  2. For XCode 6.x through 8.x (and probably 5.x), the plug in must be installed in the application at: 
    /Applications/Xcode.app/Contents/Plugins/Xcode3Core.ideplugin/Contents\
/SharedSupport/Developer/Library/Xcode/Plug-ins/
  3. For Xcode 4.x it can alternatively be installed at 
    /Library/Application Support/Developer/Shared/Xcode/Plug-ins/
  4. In your XCode project under "Other Warning Flags" remove the -Wmost option (if needed).
  5. You should now be able to select GCC 4.8 or GCC 4.9 from the compiler pop-up menu and compile code using that compiler.
  6. Note that this approach fails in El Capitan with XCode 7.x and fails when trying to link to the xerces library installed in its default location. One way to compile in El Capitan with XCode 7.x is:
    • Install GCC 4.9
    • Go to /opt/local/lib/gcc49 and create a symbolic link to the xerces library using: 
      ln -s /usr/local/lib/libxerces-c-3.1.dylib libxerces-c-3.1.dylib

      Note that the symbolic link is to a versioned xerces file and not the standard libxerces-c.dylib because linking to this link did not work. Because the xerces version is hard coded into the name, you will need to update when xerces is changed or alter the "3.1" if using a different version now. Although it should be possible to set Library Search Paths instead of creating this link, I could not get that approach to work, while the link did work.

    • Make sure to select compiler GCC 4.9 in XCode
    • Add -fopenmp to "Other C++ Flags" to make sure it is compiled with OpenMP
    • Add -lgomp to "Other Linker Flags" to link to OpenMP library.

The second approach is to build the plug in yourself (this approach can also be used to customize the plug ins downloaded above). The details (from Hammad's blog 1 and blog 2) are as follows (and for GCC 4.9 or newer, replace all uses of 4.8 with the desired version):

  1. For XCode 4.x, copy a current compiler plug in into to a /Library level plug in folder:
    • Go to XCode's plug-ins folder, which is in the application package: 
      cd /Applications/Xcode.app/Contents/PlugIns/Xcode3Core.ideplugin/Contents\
/SharedSupport/Developer/Library/Xcode/Plug-ins
    • Create a copy of "GCC 4.2.xcplugin", put it in the Xcode plugin folder (create that folder if needed), and go to its contents: 
      sudo mkdir -p "/Library/Application Support/Developer/Shared/Xcode/Plug-ins/"
sudo cp -r "GCC 4.2.xcplugin" "/Library/Application Support/Developer/Shared\
/Xcode/Plug-ins/GCC 4.8.xcplugin"
cd "/Library/Application Support/Developer/Shared/Xcode/Plug-ins/GCC 4.8.xcplugin/Contents"
  2. For XCode 5.x and newer you have to either download the plug in (see above) or copy the one mentioned in the previous step from a copy of XCode 4.x. Once copied, in must be installed in the new XCode app instead of the folder in the previous step. The correct folder is 
    /Applications/Xcode.app/Contents/PlugIns/Xcode3Core.ideplugin/Contents\
/SharedSupport/Developer/Library/Xcode/Plug-ins

    Once copied to that folder, navigate to the contents of the plug in: 

    cd /Applications/Xcode.app/Contents/PlugIns/Xcode3Core.ideplugin/Contents\
/SharedSupport/Developer/Library/Xcode/Plug-insGCC 4.8.xcplugin/Contents
  3. Using Terminal app inside the plug in, convert the binary plist into text xml and then open for editing (done with vi here, but could use another tool): 
    sudo plutil -convert xml1 Info.plist
sudo vi Info.plist
  4. Make the following changes: 
    "com.apple.xcode.compilers.gcc.42" -> "com.apple.xcode.compilers.gcc.48"
"GCC 4.2 Compiler Xcode Plug-in" -> "GCC 4.8 Compiler Xcode Plug-in"
  5. Save and convert Info.plist back to binary: 
    sudo plutil -convert binary1 Info.plist
  6. In the "Resources" folder rename two files: 
    cd Resources/
sudo mv GCC\ 4.2.xcspec GCC\ 4.8.xcspec
cd English.lproj/
sudo mv GCC\ 4.2.strings GCC\ 4.8.strings
  7. Open the "GCC 4.8.xcspec" for editing (e.g., using sudo vi) and make the changes: 
    Identifier = "com.apple.compilers.gcc.4_8";
Name = "GCC 4.8";
Description = "GNU C/C++ Compiler 4.8";
Version = "4.8";
ExecPath = "gcc-mp-4.8";
ShowInCompilerSelectionPopup = YES;
IsNoLongerSupported = NO;
  8. Down further in that file, make the following changes: 
    under Name = "GCC_ENABLE_PASCAL_STRINGS"; set DefaultValue = NO;
under Name = "GCC_CW_ASM_SYNTAX"; set DefaultValue = NO;
  9. Make other customizations, if desired.
  10. In your Xcode project under "Other Warning Flags" remove the -Wmost option (if needed).
  11. You should now be able to select GCC 4.8 from the compiler pop-up menu and compile code using that compiler.

XCode Build Settings

Compiling using a GCC compiler in Xcode requires some specific project build settings:

  1. Using GCC - these settings only work in XCode 7 or older
    • Compiler for C/C++/Objective-C: selectGCC 4.9 (or version installed), which is available when plug in is properly installed.
    • Other C Flags: add -fopenmp.
    • Other C++ Flags: add $(OTHER_CFLAGS) to have same flags used for C++ compile as well.
    • Other Linker Flags: add -lgomp.
    • Other Warning Flags: remove the -Wmost option
Retrieved from "http://osupdocs.forestry.oregonstate.edu/index.php?title=Compiling_in_MacOS_X&oldid=9935"