Difference between revisions of "Compiling in MacOS X"

From OSUPDOCS
Jump to navigation Jump to search
 
(433 intermediate revisions by the same user not shown)
Line 3: Line 3:
== Using XCode ==
== Using XCode ==


Most development of [[NairnMPM]] and [[NairnFEA]] is done in MacOS X and thus compiling on MacOS X is easy. The preferred method is to use XCode (but first you must fix its [[#MacOS X Compiler|compiler problem]]). A complete XCode project is located at
Most development of [[NairnMPM]] and [[NairnFEA]] is done in MacOS X and thus compiling on MacOS X should be easy. The preferred method is to use XCode (but first you must [[#Installing XCode|install XCode, install command line tools, and pick a compiler]]). A complete XCode project is located at


  nairn-mpm-fea/Common/Projects/NairnMPM.xcodeproj
  nairn-mpm-fea/Common/Projects/NairnMPM.xcodeproj


This project is called <tt>NairnMPM</tt>, but it includes all MPM and FEA source code in two targets named <tt>NairnMPM</tt> and <tt>NairnFEA</tt>. Once [[#Compiling Xerces on MacOS X|<tt>xerces</tt> is installed]], open the <tt>NairnMPM.xcodeproj</tt> to compile the code by:
This project is called <tt>NairnMPM</tt>, but it includes all MPM and FEA source code in two targets named <tt>NairnMPM</tt> and <tt>NairnFEA</tt> (as well as targets for [[ExtractMPM]] and [[CompareGlobal]]). Once [[#Compiling Xerces on MacOS X|<tt>xerces</tt> is installed]], open the <tt>NairnMPM.xcodeproj</tt> to compile the code by:


# Select the target to compile - either <tt>NairnMPM</tt> or <tt>NairnFEA</tt>
# Select the target to compile - either <tt>NairnMPM</tt> or <tt>NairnFEA</tt> (or [[ExtractMPM]] or [[CompareGlobal]])
# Choose build and they will compile and be saved in XCode's derived data folder.
# Choose build and they will compile and be saved in XCode's derived data folder.


If a linking error occurs, you might have conflicting architectures between the XCode settings and the [[#Compiling Xerces on MacOS X|<tt>xerces</tt> library you installed]]. To fix this problem edit <i>both</i> the project and target settings and under the &quot;Architectures&quot; section, set the &quot;Architectures&quot; option to match the architecture you used when [[#Compiling Xerces on MacOS X|installing <tt>xerces</tt>]] or recompile <tt>xerces</tt> if needed.
If a compile error occurs, you might need to [[#XCode Build Settings|change project settings]] to match you method for [[#Compiling for OpenMP|compiling for OpenMP]].
 
If a linking error occurs, you might have conflicting architectures between the XCode settings and the [[#Compiling Xerces on MacOS X|<tt>xerces</tt> library you installed]] or be missing some important C++ libraries. To fix <tt>xerces</tt> problems edit <i>both</i> the project and target settings and under the &quot;Architectures&quot; section, set the &quot;Architectures&quot; option to match the architecture you used when [[#Compiling Xerces on MacOS X|installing <tt>xerces</tt>]] or recompile <tt>xerces</tt> if needed. C++ library problems are sometimes caused by MacOS updates and might need some new [[#XCode Build Settings|project settings]].


== Compiling Xerces on MacOS X ==
== Compiling Xerces on MacOS X ==
Line 24: Line 26:
  /usr/local/include
  /usr/local/include


for the header files. These can be changed if needed by editing the project and target settings. Since revision 274 of the <code>nairn-mpm-fea</code> project, this library must be <tt>xerces 3.0</tt> or newer; prior to that revision, it had be <tt>xerces 2.8</tt> or older. These libraries can be obtained by downloading the  
for the header files. These can be changed if needed by editing the [[#XCode Build Settings|project and target settings]]. The <tt>nairn-mpm-fea</tt> project needs access to <tt>xerces 3.0</tt> or newer. This libraries can be obtained by downloading the  
[http://xml.apache.org/xerces-c/index.html xerces source code] from the [http://www.apache.org/ Apache Software Foundation] web site and then building and installing <tt>xerces</tt> with the following steps (after seeing potential [[#MacOS X Lion or Newer Command Line Tools|startup issues on Lion or newer systems]]):
[http://xml.apache.org/xerces-c/index.html xerces source code] from the [http://www.apache.org/ Apache Software Foundation] web site and then building and installing <tt>xerces</tt> with the following steps:


<ol>
<ol>
<li>Open Terminal app and navigate to the xerces source folder expanded from the downloaded file.</li>
<li>First [[#Installing XCode|install XCode and command line tools]].</li>
<li>Open Terminal app and navigate to the xerces folder expanded from the downloaded file.</li>
<li>Configure the code with the command:
<li>Configure the code with the command:
<pre>./configure CFLAGS="-arch x86_64" CXXFLAGS="-arch x86_64"</pre>
<pre>./configure CFLAGS="-arch x86_64 -arch arm64" CXXFLAGS="-arch x86_64 -arch arm64"</pre>
where the provided <tt>arch</tt> (or architecture) is the desired option for your machine. The main Mac options are <tt>i386</tt> for 32 bit Intel processors, <tt>x86_64</tt> for 64 bit Intel processors,  <tt>ppc</tt> for 32 bit PowerPC chips, or <tt>ppc64</tt> for 64 bit PowerPC64 chips. Although <tt>i386</tt> will work on any Intel processor, you should use <tt>x86_64</tt> if you have a 64 bit processor, because the code will run faster. To determine how many bits your Intel chip has, chose &quot;About This Mac&quot; from the Apple menu, click &quot;More Info...&quot; button, and look up the processor name in the hardware section. The &quot;Intel Core Solo&quot; and &quot;Intel Core Duo&quot; are 32 bit chips and most others are 64 bit chips.</li>
where the provided <tt>arch</tt> options (or architectures) are for Intel chips (<tt>x86_64</tt>) or the new Silcon M1 chips (<tt>arm64</tt>); you can pick one for your computer or include both to have a "fat" binary with both architectures. To determine your chips, choose &quot;About This Mac&quot; from the Apple menu.</li>
<li>When the configuration is done, use the following commands:
<li>When the configuration is done, use the following commands:
<pre>cd src
<pre>cd src
make
make
make install</pre>
sudo make install</pre>
These commands make the library (but not the unneeded <tt>xerces</tt> examples). The final install command after <tt>make</tt> is done, installs both the library and the header files at the default locations listed above. If the <tt>make install</tt> shows a permission error, you can solve that by using <tt>sudo make install</tt> instead and provide your administrator password.</li>
These commands make the library (but not the unneeded <tt>xerces</tt> examples). The final install command after <tt>make</tt> is done, installs both the library and the header files at the default locations listed above. It requires <tt>sudo</tt> for you to provide your administrator password needed to authenticate installation and the default location (which is <tt>/usr/local</tt>).</li>
<li>When working with <tt>nairn-mpm-fea</tt> on a new Mac, the <tt>xerces</tt> installation only needs to be done once. The only reason to repeat it is when a new <tt>xerces</tt> version is available and/or the project requires a new version for compatibility.</li>
<li>When working with <tt>nairn-mpm-fea</tt> on a new Mac, the <tt>xerces</tt> installation only needs to be done once. The only reason to repeat it is when a new <tt>xerces</tt> version is available and/or the project requires a new version for compatibility.</li>
</ol>
</ol>
Line 42: Line 45:
== MacOS X Command Line Compiling ==
== MacOS X Command Line Compiling ==


It is easy to compile on MacOS X using a command line approach (after seeing potential [[#MacOS X Lion or Newer Command Line Tools|startup issues on Lion or newer systems]] and after [[#MacOS X Compiler|updating the compiler]]). First (and first time only) install <tt>xerces</tt> and its header files as [[#Compiling Xerces on MacOS X|explained above]] and make sure they are in the specified default locations. Once <tt>xerces</tt> is ready, you can compile  [[NairnMPM]] using:
It also possible to compile on MacOS X using a command line approach. After [[#Installing XCode|installing XCode, command line tools, and a compiler]] and after [[#Compiling Xerces on MacOS X|installing <tt>xerces</tt> and its header files]], you can compile  [[NairnMPM]] using:


<pre>cd nairn-mpm-fea/NairnMPM/build
<pre>cd nairn-mpm-fea/NairnMPM/build
make SYSTEM=mac
make -j 8 SYSTEM=mac (or SYSTEM=mac-clang)
</pre>
</pre>


Line 51: Line 54:


<pre>cd nairn-mpm-fea/NairnFEA/build
<pre>cd nairn-mpm-fea/NairnFEA/build
make SYSTEM=mac
make -j 8 SYSTEM=mac (or SYSTEM=mac-clang)
</pre>
 
The <tt>-j 8</tt> option compiles in parallel where the number is number of available cores on your computer. The <tt>SYSTEM</tt> setting tells the <tt>make</tt> process what compiler to use. Use <tt>SYSTEM=mac</tt> if you have [[#Convince Apple-Installed clang to Compile for OpenMP|modified Apple's compiler]] or <tt>SYSTEM=mac-clang</tt> if you [[#Install clang-mp Compiler|installed a separate <tt>clang-mp</tt>]].
 
The above command will compile the source cade and the executables will be installed in <tt>nairn-mpm-fea/NairnMPM/input</tt> or <tt>nairn-mpm-fea/NairnFEA/input</tt>, respectively. You can use an additional <tt>make install</tt> command to copy each compiled executable to your <tt>~/bin</tt> folder if desired.
 
If the command-line compile does not work, the most likely explanations are:
 
# A problem with your compiler settings. Two options are to to edit the <tt>makefile</tt> to use your [[#Compiling for OpenMP|chosen compiler]] or to [[#Compiling Old Version of the Code|pass alternate parameters to the <tt>make</tt> command]].
# A problem with the <tt>xerces</tt> installation. You either have to install it as specified [[#Compiling Xerces on MacOS X|above]] or edit the <tt>makefile</tt> to recognize your custom installation. The process is documented in the <tt>makefile</tt> and involves editing the <tt>xercesSo</tt> and <tt>headersPath</tt> variables for your different settings.
 
=== Compiling Old Version of the Code ===
 
You can change settings used in the <tt>make</tt> process without editing the <tt>makefile</tt> by passing alternate parameters to the <tt>make</tt> command. One reason to use this approach is to compile old versions of the code. If old versions are checked out, the XCode project may not work (because of changes in XCode versions, build settings, or compiler you have installed). When this occurs, you can use command-line compile methods instead. For example, if working with current code on on Mac and the clang-mp compiler, you may be able to compile old versions using the build folders and the command
 
<pre>make CC=/opt/local/bin/clang-mp-11 CCLINK=/opt/local/bin/clang++-mp-11 \
        xercesSo=/usr/local/lib/libxerces-c.dylib \
        CFLAGS="-c -O3 -fopenmp -std=c++11 -I/opt/local/include/libomp" \
        LFLAGS="-fopenmp -lc++"
</pre>
</pre>


All source code will be compiled and the executables will be installed in <tt>nairn-mpm-fea/NairnMPM/input</tt> or <tt>nairn-mpm-fea/NairnFEA/input</tt>, respectively. You can use an additional <tt>make install</tt> command to copy each compiled executable to your <tt>~/bin</tt> folder if desired.
where <tt>CC</tt> and <tt>CCLINK</tt> refers to a <tt>clang-mp</tt> compiler and linker you have installed (old versions of this project do not use <tt>CCLINK</tt>). The other parameters might also need changing. You can open the project's <tt>makefile</tt> to see its default options for these variables and then change as needed.
 
When working with current version of the project, a better approach than using command-line arguments is to edit the <tt>makefile</tt> and pick your own default settings. Support for using a [[#Convince Apple-Installed clang to Compile for OpenMP|modifed Apple compiler]] should be built in already using the <tt>SYSTEM=mac</tt> setting. To add support for a [[#Install clang-mp Compiler|different version of <tt>clang-mp</tt>]], just follow examples used for others versions in the current <tt>makefile</tt>.


If the command-line compile does not work, the most likely explanation is a problem with the <tt>xerces</tt> installation. You either have to install it as specified [[#Compiling Xerces on MacOS X|above]] or edit the <tt>makefile</tt> to recognize your custom installation. The process is documented in the <tt>makefile</tt> and involves editing the <tt>xercesSo</tt> and <tt>headersPath</tt> variables for your different settings.
== Installing XCode ==


You can pass additional parameters to the <tt>make</tt> command to alter the compilation process. See comments in the <tt>makefile</tt> for all the latest options.
The simplest way to compiling on Mac requires that you install XCode. You can get XCode from the Mac App Store or from the Apple Developer site. It is a large install. Once installed, you then need to [[#Installing Command Line Tools|install command line tools]]. Unfortunately, the compilers provided in Apple's "Command Line Tools" do not support OpenMP used to make these code engines parallel. To compile for parallel calculations (which is important), see the two options below to [[#Compiling for OpenMP|compile for OpenMP]]. Whenever you update your MacOS or XCode, you may need to [[#Possible Changes Needed After Updating MacOS or XCode|repeat some installation steps]] before you can return to compiling the code engines.


== Using Eclipse IDE ==
The following sections have more details on installations needed for compiling the code engines.


Because the Eclipse IDE can run on multiple platforms, you can use it on Mac OS X for compiling as well. For details, see help on [[Compiling Using Eclipse IDE| compiling using Eclipse IDE]].
=== Installing Command Line Tools ===


== MacOS X Lion or Newer Command Line Tools ==
Apple used to install command line tools by default, but stopped doing that a long time ago. You now have to manually install the tools before you can compile code. These steps should work:


Apple choose not to install command line tools on Lion (or newer systems such as Mountain Lion) and makes it hard to access some very useful folders (such as your home &quot;Library&quot; folder). With these changes, you cannot compile [[#Using XCode|<tt>nairn-fea-mpm</tt> code engines]] or [[#Compiling Xerces on MacOS X|<tt>xerces</tt>]] or until you do some additional set up:
<ol>
<li>The simplest process is to open the Terminal app and enter:
<pre>xcode-select --install</pre>
This install may or may not require XCode to already be installed. If it fails try installing XCode first. If it stall fails, refer to [http://guide.macports.org/#installing.xcode MacPorts documentation] or try installing [https://brew.sh/ HomeBrew] (which I think handles command-line too installation as well.</li>
<li>You may need to agree to XCode license by using Terminal app to enter <pre>sudo xcodebuild -license</pre></li>
<li>Most tools should now be available. You should be able to make and install the [[#Compiling Xerces on MacOS X|<tt>xerces</tt>]] library.</li>
<li>To compile the code engines for parallel code, you will additionally need to [[#Compiling for OpenMP|install a compiler]] that works with OpenMP (none is provided in the command line tools).</li>
</ol>


# You probably have to install XCode (even if you do not plan to use it).
With each new system, Apple creates road blocks to keep you from using you computer for interesting programming. For now, by using a few tricks it is still possible to get around their road blocks. Hopefully these tricks will continue to work in the future.
# In XCode, use its preferences window to install &quot;Command Line Tools&quot;.
# Most tools should now be available. You should be able to make and install the [[#Compiling Xerces on MacOS X|<tt>xerces</tt>]] project and then use the provided [[#Using XCode|XCode project]] to build [[OSParticulas]], [[NairnMPM]], and [[NairnFEA]].


One issue in the XCode project is that it might not find your <tt>xerces</tt> library, even if it is in the standard location. If it is not found, the library will be in <font color="red">red</font> in the &quot;External Frameworks and Libraries&quot; folder. You have to select the library, view its properties, and then click to set the path on your computer. The problem is that you cannot navigate to the <tt>/usr/local/lib</tt> folder anymore (another new Apple &quot;feature&quot;). Here is a trick to get there:
One issue in the XCode project is that it might not find your <tt>xerces</tt> library, even if it is in the standard location. If it is not found, the library may be in <font color="red">red</font> in the &quot;External Frameworks and Libraries&quot; folder (or may not). A potential solution is to delete the reference to the library and then add it back. The problem is that you cannot navigate to the <tt>/usr/local/lib</tt> folder anymore (another new Apple &quot;feature&quot;). Here is a trick to get there:


# In the Finder, use the &quot;Go to Folder...&quot; menu command and enter &quot;<tt>/usr/local</tt>&quot;.
# In the Finder, use the &quot;Go to Folder...&quot; menu command and enter &quot;<tt>/usr/local</tt>&quot;.
# After it opens, choose the &quot;Add To Sidebar&quot; menu command. This folder will now appear in all file selection boxes.
# After it opens, choose the &quot;Add To Sidebar&quot; menu command. This folder will now appear in all file selection boxes.
# Go back to XCode and enter box to select the path the the <tt>xerces</tt> library.
# Go back to XCode and use command to add files for adding the <tt>xerces</tt> library.
# Go through your new &quot;local&quot; folder and select the <tt>xerces</tt> library file in <tt>/usr/local/lib</tt>.
# Go through your new &quot;local&quot; folder and select the <tt>xerces</tt> library file in <tt>/usr/local/lib</tt>.


Apple seems to creating road blocks to keep you from using you computer for interesting programming. With a few tricks it is still possible to get around their road blocks. Hopefully these tricks will continue to work in the future.
=== Compiling for OpenMP ===
 
Although Apple uses <tt>clang</tt> for compiling in XCode, for reasons that are hard to justify, they opted to disable compiling with OpenMP (<i>i.e.</i>, they disabled the <tt>-fopenmp</tt> compiler option). They further did not include <tt>libomp.dylib</tt>, which is needed to run software written for OpenMP. Because doing parallel computations is essential for large calculations, some method to use OpenMP is needed. Two possible solutions are given in this section.
 
==== Convince Apple-Installed <tt>clang</tt> to Compile for OpenMP ====
 
The approach with minimal installation is to override Apple's disabling of the <tt>-fopenmp</tt> option to compile for OpenMP anyway. This approach uses the following steps:
 
<ol>
<li>The only extra installation required is a signed copy of the <tt>libomp.dylib</tt>. Fortunately. a member of the R project has made this installation easy. First check your version of XCode. Next, look up file name you need in the following table:
{| class="wikitable"
|-
! XCode Version !! File name to download
|-
| align="center"  | 15.0+ || openmp-16.0.4-darwin20-Release.tar.gz
|-
| align="center"  | 14.3.x || openmp-15.0.7-darwin20-Release.tar.gz
|-
| align="center"  | 14.0 to 14.2 || openmp-14.0.6-darwin20-Release.tar.gz
|-
| align="center"  | 13.3 to 13.4.1 || openmp-13.0.0-darwin21-Release.tar.gz
|-
| align="center"  | 13.0 to 13.2.1 || openmp-12.0.1-darwin20-Release.tar.gz
|}
If you have a newer version of XCode, you can check the [https://mac.r-project.org/openmp/ R project posting] for a new file name. If you have an older version of XCode, now is a good time to update your Mac.</li>
<li>Once you have the file name, download it from <tt>r-project.org</tt> in the Terminal app using:
<pre>curl -O https://mac.r-project.org/openmp/openmp-15.0.7-darwin20-Release.tar.gz</pre>
replacing the file name with the one you need for your version of XCode. If you visit the [https://mac.r-project.org/openmp/ R project posting], do not download with the links in that web site because Apple will quarantine all files rendering the library unusable without more work.
</li>
<li>Once downloaded, extract and install <tt>libomp</tt> files with
<pre>sudo tar fvxz openmp-15.0.7-darwin20-Release.tar.gz -C /</pre>
being sure to replace the file name in this command with the one downloaded in the previous step.</li>
<li>The <tt>-C /</tt> option in the previous step extracts the files directly to your root <tt>/usr/local</tt> folder rather a folder next to the downloaded file. When done, the following files should be installed:
<pre>/usr/local/lib/libomp.dylib
/usr/local/include/ompt.h
/usr/local/include/omp.h
/usr/local/include/omp-tools.h</pre>
</li>
</ol>
 
Once these steps are done, you are ready to [[#Using XCode|compile in XCode]] or [[#MacOS X Command Line Compiling|compile on a command line]] (using the <tt>SYSTEM=mac</tt> option). If an XCode compile has problems, you may have adjust some of the [[#Project Setting for Apple's Compiler|XCode project settings]] for this compiler option.
 
According to the [https://mac.r-project.org/openmp/ R project posting], this approach works now, but " may break at any time." Furthermore, each time you upgrade XCode, you may need to install a signed and updated version of <tt>libomp.dylib</tt>. If for some reason the method stops working or an updated <tt>libomp.dylib</tt> cannot be found, you can switch to an alternate method that involves installing a separate compiler (and is explain in the next section).
 
==== Install <tt>clang-mp</tt> Compiler ====
 
An alternate approach to compiling parallel code using OpenMP is to install a separate <tt>clang-mp</tt> compiler. One way to install this compiler is to use [http://www.macports.org MacPorts] as follows:
 
<ol>
<li>If not already installed, install [http://www.macports.org MacPorts]. Be aware that it might need to be [[#Possible Changes Needed After Updating MacOS or XCode|reinstalled after updating your MacOS]].</li>
<li>Open the Terminal app and enter
<pre>sudo port selfupdate
sudo port install clang-11 ld64 +ld64_xcode</pre>
The first command above is only needed to make sure your MacPorts is up to date (you might need to reinstall instead of <tt>selfupdate</tt>  [[#Possible Changes Needed After Updating MacOS or XCode|after updating your MacOS]]). The second step installs the compiler version requested (change "11" to version you want)
</li>
<li>To verify it installed, check in Terminal app:
<pre>which clang-mp-11
/opt/local/bin/clang-mp-11</pre>
If the <tt>which</tt> command did not return the installed location, an error occurred during install.
</ol>
 
The above install usually works. Some issues have occurred in MacOS Sonoma and new issues might happen in future MacOS versions. Here are some known issues:
 
# A MacPorts <tt>install</tt> command in MacOS Sonoma reports that <tt>clang-mp</tt> versions 11, 12, and 13 are "known to fail." It appears these version can still be installed, but the <tt>sudo port install</tt> may not finish. If it exits without a binary, repeat the <tt>install</tt> command until it (hopefully) finishes. Each repeat picks up where it left off. If it hangs up while installing, force exit with Control-C and repeat again. If it eventually finishes, the install seems to work. I needed three repeats for one successful install of <tt>clang-mp-11</tt>.
# Installing <tt>clang-mp</tt> versions 14 or newer in MacOS Sonoma does not say "known to fail", but the first <tt>install</tt> command may exit with an unexpected "permission denied" error. Repeating the <tt>install</tt> command then picks up where it left off and finishes the install.
# Although these versions get installed, you may need to verify the [[#Project Settings for clang-mp|XCode project settings]] match both your installed version of <tt>clang-mp</tt> and your choice to use a separate compiler. A new issue occurs in MacOS Sonoma, but that is corrected with a new [[#Project Settings for clang-mp|runpath project setting]]. Future MacOS's might cause more problems and some might be fixable with new project settings.
 
You can install multiple versions of <tt>clang-mp</tt>, which can be useful if working with multiple checkouts of this project. More likely, you only need one version. If you settle on one version, you can remove older ones with
 
sudo port uninstall --follow-dependents clang-xx
 
where <tt>xx</tt> is the version you want to uninstall.
 
=== XCode Build Settings ===
 
The downloaded XCode project will have project settings for compiling with [[#Convince Apple-Installed clang to Compile for OpenMP|Apple's compiler]] or with an [[#Install clang-mp Compiler|installed <tt>clang-mp</tt> compiler]]. To find out which one, open the project in XCode, find the "Build Settings" for the <tt>NairnMPM</tt> or <tt>NairnFEA</tt> target, and scroll the to bottom to see "User-Defined" settings. If there are no "User-Defined" settings, the project is set up to use [[#Convince Apple-Installed clang to Compile for OpenMP|Apple's compiler]]. If there are "User-Defined" settings, the <tt>CC</tt> setting specifies the <tt>clang-mp</tt> version being used. The next to sections explain how to change project settings if you want to use a different compiler or change to a different <tt>clang-mp</tt> version.
 
==== Project Setting for Apple's Compiler ====
 
Compiling using [[#Convince Apple-Installed clang to Compile for OpenMP|Apple's compiler]] in XCode requires these specific project build settings. Change as needed by editing the build settings for the <tt>NairnMPM</tt> and <tt>NairnFEA</tt> targets (both need to be edited):
 
<ul>
<li><b>Compiler for C/C++/Objective-C</b>: select the default compiler.</li>
<li><b>Other Linker Flags</b>: add <tt>-lomp</tt> (replacing <tt>-fopenmp</tt> if there).</li>
<li><b>Runpath Search Paths</b>: remove <tt>/opt/local/libexec/llvm-17/lib</tt> (or with other clang version number) if it is there.</li>
<li><b>Header Search Paths</b>: remove <tt>/opt/local/include/libomp</tt> if it is there</li>
<li><b>Other C Flags</b>: add <tt>-Xclang -fopenmp</tt> together as a single flag (replacing <tt>-fopenmp</tt> if there).</li>
<li><b>Other C++ Flags</b>: add <tt>$(OTHER_CFLAGS)</tt> to have same flags used for C++ compile as well.</li>
<li>Scroll to bottom of build settings to see the "User-Defined" settings. If <b>CC</b>, <b>CLANG_CXX_LIBRARY</b>, and <b>LDPLUSPLUS</b> are there, delete them be selecting and using the delete key.</li>
</ul>


== MacOS X Compiler ==
==== Project Settings for <tt>clang-mp</tt> ====


The "Command Line Tools" installed by XCode include a modified GCC 4.2 compiler (and rumors suggest Apple will not update to more modern GCC in favor of their own compiler tools). Unfortunately, this compiler does not work well with OpenMP used to make the code engines parallel. On some Macs, code compiled with the default compiler will randomly crash when run in parallel. On others, the code will not even compile (getting a load area due to OpenMP code). The solution is to install and use a newer version of GCC. The steps are:
Compiling using a [[#Install clang-mp Compiler|<tt>clang-mp</tt> compiler]] in XCode requires these specific project build settings. Change as needed by editing the build settings for the <tt>NairnMPM</tt> and <tt>NairnFEA</tt> targets (both need to be edited):


<ul>
<ul>
<li>Install newer version of GCC, such as GCC 4.8. A good way is to install using [http://www.macports.org MacPorts]. Once that is tool is installed, the installation of GCC is easy, as explained in this [http://www.ficksworkshop.com/blog/14-coding/65-installing-gcc-on-mac blog]. In brief, all you need is to open Terminal app and enter
<li><b>Compiler for C/C++/Objective-C</b>: select the default compiler.</li>
<pre>sudo port selfupdate
<li><b>Other Linker Flags</b>: add <tt>-fopenmp</tt> (replacing <tt>-lomp</tt> if there).</li>
sudo port install gcc48
<li><b>Runpath Search Paths</b>: add <tt>/opt/local/libexec/llvm-17/lib</tt> (using clang version number you installed). This setting was not necessary until <tt>clang-mp-14</tt> (or maybe it was caused by MacOS Sonoma). Without this setting the executable fails to load the standard <tt>C++</tt> libraries. If you compile and run without this setting, you will get an error saying library not found. If you get that error, make sure this setting is correct.</li>
</pre>The first is only need to make sure your MacPorts is up to date.</li>
<li><b>Other C Flags</b>: add <tt>-fopenmp</tt> (replacing <tt>-Xclang -fopenmp</tt> if there).</li>
<li>The first step will install the compiler at
<li><b>Other C++ Flags</b>: add <tt>$(OTHER_CFLAGS)</tt> to have same flags used for C++ compile as well.</li>
<pre>/opt/local/bin/g++-mp-4.8
<li><b>Header Search Paths</b>: add <tt>/opt/local/include/libomp</tt>
</pre>For [[#MacOS X Command Line Compiling|command line compiling]], the first step is enough. The <tt>makefile</tt>s in the build folders assumes this path when command-line compiling on a Mac.</li>
<li>Scroll to bottom of build settings to see the "User-Defined" settings. If <b>CC</b>, <b>CLANG_CXX_LIBRARY</b>, and <b>LDPLUSPLUS</b> are already there, make sure the <tt>clang-mp</tt> version numbers in <b>CC</b> and <b>LDPLUSPLUS</b> corresponds to one you have [[#Install clang-mp Compiler|installed]] or edit them to match your installed version. If any "User-Defined" settings are missible, add them with:
<li>To use this new compiler in XCode, you have to hack XCode to support it. The process is explained on this [http://hamelot.co.uk/programming/add-custom-compiler-to-xcode/ blog]. Here are the steps from that blog (see the blog if you need more detail) done using the Terminal app:
<ol>
<ol>
<li>Go to XCode's plug-ins folder, which in the application package:
<li>Use menu command Editor&rarr;Add Build Setting&rarr;Add User-Defined Setting and add setting for <b>CC</b> with value <b>/opt/local/bin/clang-mp-11</b> (using clang version number you installed).</li>
<pre>cd /Applications/Xcode.app/Contents/PlugIns/Xcode3Core.ideplugin/Contents\
<li>Use menu command Editor&rarr;Add Build Setting&rarr;Add User-Defined Setting and add setting for <b>LDPLUSPLUS</b> with value <b>/opt/local/bin/clang++-mp-11</b> (using clang version number you installed).</li>
/SharedSupport/Developer/Library/Xcode/Plug-ins
<li>Use menu command Editor&rarr;Add Build Setting&rarr;Add User-Defined Setting and add setting for <b>CLANG_CXX_LIBRARY</b> with value <b>libc++</b>.</li>
</pre></li>
</ol>
<li>Create a copy of "GCC 4.2.xcplugin", put it in the Xcode plugin folder (create that folder if needed), and go to its contents:
<pre>sudo mkdir -p "/Library/Application Support/Developer/Shared/Xcode/Plug-ins/"
sudo cp -r "GCC 4.2.xcplugin" "/Library/Application Support/Developer/Shared\
/Xcode/Plug-ins/GCC 4.8.xcplugin"
cd "/Library/Application Support/Developer/Shared/Xcode/Plug-ins/GCC 4.8.xcplugin/Contents"
</pre></li>
<li>Convert the binary <tt>plist</tt> into text <tt>xml</tt> and then open for editing (done with <tt>vi</tt> here, but could use another tool):
<pre>sudo plutil -convert xml1 Info.plist
sudo vi Info.plist
</pre></li>
<li>Make the following changes:
<pre>"com.apple.xcode.compilers.gcc.42" -> "com.apple.xcode.compilers.gcc.48"
"GCC 4.2 Compiler Xcode Plug-in" -> "GCC 4.8 Compiler Xcode Plug-in"
</pre></li>
<li>Save and convert <tt>Info.plist</tt> back to binary:
<pre>sudo plutil -convert binary1 Info.plist
</pre></li>
<li>In the "Resources" folder rename two files:
<pre>cd Resources/
sudo mv GCC\ 4.2.xcspec GCC\ 4.8.xcspec
cd English.lproj/
sudo mv GCC\ 4.2.strings GCC\ 4.8.strings
</pre></li>
<li>Open the "GCC 4.8.xcspec" for editing (''e.g.'', using <tt>sudo vi</tt>) and make the changes:
<pre>Identifier = "com.apple.compilers.gcc.4_8";
Name = "GCC 4.8";
Description = "GNU C/C++ Compiler 4.8";
Version = "4.8";
ExecPath = "gcc-mp-4.8";
ShowInCompilerSelectionPopup = YES;
IsNoLongerSupported = NO;
</pre></li>
<li>Down further in that file, make the following changes:
<pre>under Name = "GCC_ENABLE_PASCAL_STRINGS"; set DefaultValue = NO;
under Name = "GCC_CW_ASM_SYNTAX"; set DefaultValue = NO;
</pre>
<li>In your Xcode project under "Other Warning Flags" remove the <tt>-Wmost</tt> option.
</li>
</li>
</ol>
</ul>
</ul>
=== Possible Changes Needed After Updating MacOS or XCode ===
Unfortunately, Apple does not do a very good job of preserving your prior settings after a major upgrade and you will often need a complete reinstall of anything installed using MacPorts. Some things that might be needed with each MacOS upgrade are:
# Get the latest version of XCode (from developer web site or app store).
# Reinstall command line tools in Terminal app by entering <tt>xcode-select --install</tt>
# Agree to XCode license in Terminal app using <tt>sudo xcodebuild -license</tt> (and provide your admin password).
# If you are using [[#Convince Apple-Installed clang to Compile for OpenMP|Apple's compiler]] you may need to update the <tt>libomp.dylib</tt> library for your new version of XCode.
# If you are using MacPorts, you may need to uninstall it, install MacPorts for new MacOS, and then reinstall all previously installed ports. The process is explained in [https://trac.macports.org/wiki/Migration Migrating a MacPorts Installation]. Sometimes a "clean" install is best, especially after a major MacOS update. See [https://trac.macports.org/wiki/previous links] for details, but here is a brief outline of the process:
## [https://guide.macports.org/chunked/installing.macports.uninstalling.html Uninstall all of MacPorts]
## [https://guide.macports.org/chunked/installing.macports.html Install MacPorts for the new MacOS]
## Renstall latest versions of all ports needed for this project ([[#Compiling for OpenMP|clang-mp]] and maybe [[OSParticulas#Installing svn in MacOS|subversion too]]). If you use different ports for other projects, you might want to use one of the advanced approaches on the [http://www.macports.org MacPorts] web site when migrating to new a new MacOS instead.
# To compile in XCode, you may need more changes (but hard to predict what Apple will require next).

Latest revision as of 14:37, 9 April 2024

This page explains several methods to compile code engines using MacOS X.

Using XCode

Most development of NairnMPM and NairnFEA is done in MacOS X and thus compiling on MacOS X should be easy. The preferred method is to use XCode (but first you must install XCode, install command line tools, and pick a compiler). A complete XCode project is located at

nairn-mpm-fea/Common/Projects/NairnMPM.xcodeproj

This project is called NairnMPM, but it includes all MPM and FEA source code in two targets named NairnMPM and NairnFEA (as well as targets for ExtractMPM and CompareGlobal). Once xerces is installed, open the NairnMPM.xcodeproj to compile the code by:

  1. Select the target to compile - either NairnMPM or NairnFEA (or ExtractMPM or CompareGlobal)
  2. Choose build and they will compile and be saved in XCode's derived data folder.

If a compile error occurs, you might need to change project settings to match you method for compiling for OpenMP.

If a linking error occurs, you might have conflicting architectures between the XCode settings and the xerces library you installed or be missing some important C++ libraries. To fix xerces problems edit both the project and target settings and under the "Architectures" section, set the "Architectures" option to match the architecture you used when installing xerces or recompile xerces if needed. C++ library problems are sometimes caused by MacOS updates and might need some new project settings.

Compiling Xerces on MacOS X

Before you can compile and run the project, however, you will need an installed version of the xerces library and a copy of the xerces header files. These compiling instructions assume they are installed in the default locations for MacOS X or at:

/usr/local/lib/libxerces-c.dylib

for the library and at

/usr/local/include

for the header files. These can be changed if needed by editing the project and target settings. The nairn-mpm-fea project needs access to xerces 3.0 or newer. This libraries can be obtained by downloading the xerces source code from the Apache Software Foundation web site and then building and installing xerces with the following steps:

  1. First install XCode and command line tools.
  2. Open Terminal app and navigate to the xerces folder expanded from the downloaded file.
  3. Configure the code with the command:
    ./configure CFLAGS="-arch x86_64 -arch arm64" CXXFLAGS="-arch x86_64 -arch arm64"
    where the provided arch options (or architectures) are for Intel chips (x86_64) or the new Silcon M1 chips (arm64); you can pick one for your computer or include both to have a "fat" binary with both architectures. To determine your chips, choose "About This Mac" from the Apple menu.
  4. When the configuration is done, use the following commands:
    cd src
    make
    sudo make install
    These commands make the library (but not the unneeded xerces examples). The final install command after make is done, installs both the library and the header files at the default locations listed above. It requires sudo for you to provide your administrator password needed to authenticate installation and the default location (which is /usr/local).
  5. When working with nairn-mpm-fea on a new Mac, the xerces installation only needs to be done once. The only reason to repeat it is when a new xerces version is available and/or the project requires a new version for compatibility.

MacOS X Command Line Compiling

It also possible to compile on MacOS X using a command line approach. After installing XCode, command line tools, and a compiler and after installing xerces and its header files, you can compile NairnMPM using:

cd nairn-mpm-fea/NairnMPM/build
make -j 8 SYSTEM=mac (or SYSTEM=mac-clang)

and compile NairnFEA using:

cd nairn-mpm-fea/NairnFEA/build
make -j 8 SYSTEM=mac (or SYSTEM=mac-clang)

The -j 8 option compiles in parallel where the number is number of available cores on your computer. The SYSTEM setting tells the make process what compiler to use. Use SYSTEM=mac if you have modified Apple's compiler or SYSTEM=mac-clang if you installed a separate clang-mp.

The above command will compile the source cade and the executables will be installed in nairn-mpm-fea/NairnMPM/input or nairn-mpm-fea/NairnFEA/input, respectively. You can use an additional make install command to copy each compiled executable to your ~/bin folder if desired.

If the command-line compile does not work, the most likely explanations are:

  1. A problem with your compiler settings. Two options are to to edit the makefile to use your chosen compiler or to pass alternate parameters to the make command.
  2. A problem with the xerces installation. You either have to install it as specified above or edit the makefile to recognize your custom installation. The process is documented in the makefile and involves editing the xercesSo and headersPath variables for your different settings.

Compiling Old Version of the Code

You can change settings used in the make process without editing the makefile by passing alternate parameters to the make command. One reason to use this approach is to compile old versions of the code. If old versions are checked out, the XCode project may not work (because of changes in XCode versions, build settings, or compiler you have installed). When this occurs, you can use command-line compile methods instead. For example, if working with current code on on Mac and the clang-mp compiler, you may be able to compile old versions using the build folders and the command

make CC=/opt/local/bin/clang-mp-11 CCLINK=/opt/local/bin/clang++-mp-11 \
        xercesSo=/usr/local/lib/libxerces-c.dylib \
        CFLAGS="-c -O3 -fopenmp -std=c++11 -I/opt/local/include/libomp" \
        LFLAGS="-fopenmp -lc++"

where CC and CCLINK refers to a clang-mp compiler and linker you have installed (old versions of this project do not use CCLINK). The other parameters might also need changing. You can open the project's makefile to see its default options for these variables and then change as needed.

When working with current version of the project, a better approach than using command-line arguments is to edit the makefile and pick your own default settings. Support for using a modifed Apple compiler should be built in already using the SYSTEM=mac setting. To add support for a different version of clang-mp, just follow examples used for others versions in the current makefile.

Installing XCode

The simplest way to compiling on Mac requires that you install XCode. You can get XCode from the Mac App Store or from the Apple Developer site. It is a large install. Once installed, you then need to install command line tools. Unfortunately, the compilers provided in Apple's "Command Line Tools" do not support OpenMP used to make these code engines parallel. To compile for parallel calculations (which is important), see the two options below to compile for OpenMP. Whenever you update your MacOS or XCode, you may need to repeat some installation steps before you can return to compiling the code engines.

The following sections have more details on installations needed for compiling the code engines.

Installing Command Line Tools

Apple used to install command line tools by default, but stopped doing that a long time ago. You now have to manually install the tools before you can compile code. These steps should work:

  1. The simplest process is to open the Terminal app and enter:
    xcode-select --install
    This install may or may not require XCode to already be installed. If it fails try installing XCode first. If it stall fails, refer to MacPorts documentation or try installing HomeBrew (which I think handles command-line too installation as well.
  2. You may need to agree to XCode license by using Terminal app to enter
    sudo xcodebuild -license
  3. Most tools should now be available. You should be able to make and install the xerces library.
  4. To compile the code engines for parallel code, you will additionally need to install a compiler that works with OpenMP (none is provided in the command line tools).

With each new system, Apple creates road blocks to keep you from using you computer for interesting programming. For now, by using a few tricks it is still possible to get around their road blocks. Hopefully these tricks will continue to work in the future.

One issue in the XCode project is that it might not find your xerces library, even if it is in the standard location. If it is not found, the library may be in red in the "External Frameworks and Libraries" folder (or may not). A potential solution is to delete the reference to the library and then add it back. The problem is that you cannot navigate to the /usr/local/lib folder anymore (another new Apple "feature"). Here is a trick to get there:

  1. In the Finder, use the "Go to Folder..." menu command and enter "/usr/local".
  2. After it opens, choose the "Add To Sidebar" menu command. This folder will now appear in all file selection boxes.
  3. Go back to XCode and use command to add files for adding the xerces library.
  4. Go through your new "local" folder and select the xerces library file in /usr/local/lib.

Compiling for OpenMP

Although Apple uses clang for compiling in XCode, for reasons that are hard to justify, they opted to disable compiling with OpenMP (i.e., they disabled the -fopenmp compiler option). They further did not include libomp.dylib, which is needed to run software written for OpenMP. Because doing parallel computations is essential for large calculations, some method to use OpenMP is needed. Two possible solutions are given in this section.

Convince Apple-Installed clang to Compile for OpenMP

The approach with minimal installation is to override Apple's disabling of the -fopenmp option to compile for OpenMP anyway. This approach uses the following steps:

  1. The only extra installation required is a signed copy of the libomp.dylib. Fortunately. a member of the R project has made this installation easy. First check your version of XCode. Next, look up file name you need in the following table:
    XCode Version File name to download
    15.0+ openmp-16.0.4-darwin20-Release.tar.gz
    14.3.x openmp-15.0.7-darwin20-Release.tar.gz
    14.0 to 14.2 openmp-14.0.6-darwin20-Release.tar.gz
    13.3 to 13.4.1 openmp-13.0.0-darwin21-Release.tar.gz
    13.0 to 13.2.1 openmp-12.0.1-darwin20-Release.tar.gz
    If you have a newer version of XCode, you can check the R project posting for a new file name. If you have an older version of XCode, now is a good time to update your Mac.
  2. Once you have the file name, download it from r-project.org in the Terminal app using:
    curl -O https://mac.r-project.org/openmp/openmp-15.0.7-darwin20-Release.tar.gz

    replacing the file name with the one you need for your version of XCode. If you visit the R project posting, do not download with the links in that web site because Apple will quarantine all files rendering the library unusable without more work.

  3. Once downloaded, extract and install libomp files with
    sudo tar fvxz openmp-15.0.7-darwin20-Release.tar.gz -C /
    being sure to replace the file name in this command with the one downloaded in the previous step.
  4. The -C / option in the previous step extracts the files directly to your root /usr/local folder rather a folder next to the downloaded file. When done, the following files should be installed:
    /usr/local/lib/libomp.dylib
    /usr/local/include/ompt.h
    /usr/local/include/omp.h
    /usr/local/include/omp-tools.h

Once these steps are done, you are ready to compile in XCode or compile on a command line (using the SYSTEM=mac option). If an XCode compile has problems, you may have adjust some of the XCode project settings for this compiler option.

According to the R project posting, this approach works now, but " may break at any time." Furthermore, each time you upgrade XCode, you may need to install a signed and updated version of libomp.dylib. If for some reason the method stops working or an updated libomp.dylib cannot be found, you can switch to an alternate method that involves installing a separate compiler (and is explain in the next section).

Install clang-mp Compiler

An alternate approach to compiling parallel code using OpenMP is to install a separate clang-mp compiler. One way to install this compiler is to use MacPorts as follows:

  1. If not already installed, install MacPorts. Be aware that it might need to be reinstalled after updating your MacOS.
  2. Open the Terminal app and enter
    sudo port selfupdate
    sudo port install clang-11 ld64 +ld64_xcode

    The first command above is only needed to make sure your MacPorts is up to date (you might need to reinstall instead of selfupdate after updating your MacOS). The second step installs the compiler version requested (change "11" to version you want)

  3. To verify it installed, check in Terminal app:
    which clang-mp-11
    /opt/local/bin/clang-mp-11

    If the which command did not return the installed location, an error occurred during install.

The above install usually works. Some issues have occurred in MacOS Sonoma and new issues might happen in future MacOS versions. Here are some known issues:

  1. A MacPorts install command in MacOS Sonoma reports that clang-mp versions 11, 12, and 13 are "known to fail." It appears these version can still be installed, but the sudo port install may not finish. If it exits without a binary, repeat the install command until it (hopefully) finishes. Each repeat picks up where it left off. If it hangs up while installing, force exit with Control-C and repeat again. If it eventually finishes, the install seems to work. I needed three repeats for one successful install of clang-mp-11.
  2. Installing clang-mp versions 14 or newer in MacOS Sonoma does not say "known to fail", but the first install command may exit with an unexpected "permission denied" error. Repeating the install command then picks up where it left off and finishes the install.
  3. Although these versions get installed, you may need to verify the XCode project settings match both your installed version of clang-mp and your choice to use a separate compiler. A new issue occurs in MacOS Sonoma, but that is corrected with a new runpath project setting. Future MacOS's might cause more problems and some might be fixable with new project settings.

You can install multiple versions of clang-mp, which can be useful if working with multiple checkouts of this project. More likely, you only need one version. If you settle on one version, you can remove older ones with

sudo port uninstall --follow-dependents clang-xx

where xx is the version you want to uninstall.

XCode Build Settings

The downloaded XCode project will have project settings for compiling with Apple's compiler or with an installed clang-mp compiler. To find out which one, open the project in XCode, find the "Build Settings" for the NairnMPM or NairnFEA target, and scroll the to bottom to see "User-Defined" settings. If there are no "User-Defined" settings, the project is set up to use Apple's compiler. If there are "User-Defined" settings, the CC setting specifies the clang-mp version being used. The next to sections explain how to change project settings if you want to use a different compiler or change to a different clang-mp version.

Project Setting for Apple's Compiler

Compiling using Apple's compiler in XCode requires these specific project build settings. Change as needed by editing the build settings for the NairnMPM and NairnFEA targets (both need to be edited):

  • Compiler for C/C++/Objective-C: select the default compiler.
  • Other Linker Flags: add -lomp (replacing -fopenmp if there).
  • Runpath Search Paths: remove /opt/local/libexec/llvm-17/lib (or with other clang version number) if it is there.
  • Header Search Paths: remove /opt/local/include/libomp if it is there
  • Other C Flags: add -Xclang -fopenmp together as a single flag (replacing -fopenmp if there).
  • Other C++ Flags: add $(OTHER_CFLAGS) to have same flags used for C++ compile as well.
  • Scroll to bottom of build settings to see the "User-Defined" settings. If CC, CLANG_CXX_LIBRARY, and LDPLUSPLUS are there, delete them be selecting and using the delete key.

Project Settings for clang-mp

Compiling using a clang-mp compiler in XCode requires these specific project build settings. Change as needed by editing the build settings for the NairnMPM and NairnFEA targets (both need to be edited):

  • Compiler for C/C++/Objective-C: select the default compiler.
  • Other Linker Flags: add -fopenmp (replacing -lomp if there).
  • Runpath Search Paths: add /opt/local/libexec/llvm-17/lib (using clang version number you installed). This setting was not necessary until clang-mp-14 (or maybe it was caused by MacOS Sonoma). Without this setting the executable fails to load the standard C++ libraries. If you compile and run without this setting, you will get an error saying library not found. If you get that error, make sure this setting is correct.
  • Other C Flags: add -fopenmp (replacing -Xclang -fopenmp if there).
  • Other C++ Flags: add $(OTHER_CFLAGS) to have same flags used for C++ compile as well.
  • Header Search Paths: add /opt/local/include/libomp
  • Scroll to bottom of build settings to see the "User-Defined" settings. If CC, CLANG_CXX_LIBRARY, and LDPLUSPLUS are already there, make sure the clang-mp version numbers in CC and LDPLUSPLUS corresponds to one you have installed or edit them to match your installed version. If any "User-Defined" settings are missible, add them with:
    1. Use menu command Editor→Add Build Setting→Add User-Defined Setting and add setting for CC with value /opt/local/bin/clang-mp-11 (using clang version number you installed).
    2. Use menu command Editor→Add Build Setting→Add User-Defined Setting and add setting for LDPLUSPLUS with value /opt/local/bin/clang++-mp-11 (using clang version number you installed).
    3. Use menu command Editor→Add Build Setting→Add User-Defined Setting and add setting for CLANG_CXX_LIBRARY with value libc++.

Possible Changes Needed After Updating MacOS or XCode

Unfortunately, Apple does not do a very good job of preserving your prior settings after a major upgrade and you will often need a complete reinstall of anything installed using MacPorts. Some things that might be needed with each MacOS upgrade are:

  1. Get the latest version of XCode (from developer web site or app store).
  2. Reinstall command line tools in Terminal app by entering xcode-select --install
  3. Agree to XCode license in Terminal app using sudo xcodebuild -license (and provide your admin password).
  4. If you are using Apple's compiler you may need to update the libomp.dylib library for your new version of XCode.
  5. If you are using MacPorts, you may need to uninstall it, install MacPorts for new MacOS, and then reinstall all previously installed ports. The process is explained in Migrating a MacPorts Installation. Sometimes a "clean" install is best, especially after a major MacOS update. See links for details, but here is a brief outline of the process:
    1. Uninstall all of MacPorts
    2. Install MacPorts for the new MacOS
    3. Renstall latest versions of all ports needed for this project (clang-mp and maybe subversion too). If you use different ports for other projects, you might want to use one of the advanced approaches on the MacPorts web site when migrating to new a new MacOS instead.
  6. To compile in XCode, you may need more changes (but hard to predict what Apple will require next).