Browse Source

Add build tooling

nosoop 1 year ago
parent
commit
9dd0d1d89b
5 changed files with 427 additions and 0 deletions
  1. 9 0
      .gitignore
  2. 94 0
      BUILD.md
  3. 125 0
      configure.py
  4. 183 0
      misc/ninja_syntax.py
  5. 16 0
      misc/spcomp_util.py

+ 9 - 0
.gitignore

@@ -3,3 +3,12 @@
 
 *.smx
 
+# Ignore ninja build stuff
+# (These should be handled with configure.py)
+**/__pycache__
+build/
+build.ninja
+
+# Ignore personal configuration files copied from contrib
+/modd.conf
+/uploader.ini

+ 94 - 0
BUILD.md

@@ -0,0 +1,94 @@
+# SourceMod Ninja Project Build Instructions
+
+This project uses the [Ninja Project Template for SourceMod Plugins][].
+
+The following is documentation of build steps required for the repository at the time this
+project was generated; the source template may be different in the future.
+
+[Ninja Project Template for SourceMod Plugins]: https://github.com/nosoop/NinjaBuild-SMPlugin
+
+## Prerequisites
+
+A few things are needed for developing with this environment:
+
+- A familiarity with command line tooling.
+    - An understanding of calling programs and changing directories will do.
+- The ninja build system.
+    - It's small, fast, cross-platform, and isn't tied to any particular set of tools.
+- Python 3.6 or newer.
+    - Used to detect our compiler and write out the build script for ninja.
+- A clean copy of the [SourceMod][] compiler.  It should not contain any third-party includes.
+    - Which version you'll need depends on the project, but assume latest stable if not
+    specified.
+    - An untouched compiler directory ensures build consistency by not polluting it with custom
+    files — all non built-in dependencies should be encapsulated in the project
+    repository (by adding the dependency as a submodule or copying the include files directly).
+    - You only need the `addons/sourcemod/scripting/` directory from the SourceMod package.
+    - The scripting directory doesn't need to be in `%PATH%` / `$PATH`; the script provides
+    `--spcomp-dir`.  This also allows you to quickly switch between compiler / SourceMod
+    versions.
+    - Do not add / commit the compiler and core includes into your project; the clean compiler
+    can be shared between projects and updated independently from your project.  However, *do*
+    add third-party includes into your project.
+
+You only need to install these once, but make sure to skim over the `README.md` in case other
+projects using this project template require additional software dependencies.
+
+<details>
+<summary>Expand to see instructions for installing dependencies</summary>
+
+1. Install `ninja`.
+    - You can download the latest version for Windows / Mac / Linux from the [ninja releases][]
+    page and install it into your path.
+    - With [Scoop][] on Windows, use `scoop install ninja`.
+    - With Debian and Debian-based distributions like Ubuntu, `apt install ninja-build` will get
+    you the distro's version, which may be a few versions behind current.  That should be fine
+    enough in most cases.
+2. Install Python 3.
+    - You can download and install it manually from [the official site][Python].
+    - With [Scoop][], `scoop install python`.
+    - With Debian-based distributions, `apt install python3`.
+3. Download the [SourceMod][] compiler.
+    - On Linux, both 32- and 64-bit versions of `spcomp` are supported by the build script; you
+    do not need to install 32-bit compatibility libraries on your build machine.
+
+</details>
+
+[ninja releases]: https://github.com/ninja-build/ninja/releases
+[Python]: https://www.python.org/
+[Scoop]: https://scoop.sh/
+[SourceMod]: https://www.sourcemod.net/
+
+## Building
+
+The tl;dr is that you should be able to build any git-based project in this format with the
+following commands:
+
+    git clone --recurse-submodules ${repo}
+    # cd into repo
+    python3 configure.py --spcomp-dir ${dir}
+    ninja
+
+Detailed explanation:
+
+1. Clone the repository and any git repositories it depends on, then navigate to it.
+2. Run `python3 configure.py --spcomp-dir ${dir}` within the project root, where `${dir}` is a
+directory containing the SourcePawn compiler (`spcomp`) and SourceMod's base include files.
+This will create the `build.ninja` script.
+    - You may need to use `python3.8` or some other variant of the executable name, depending on
+    your environment.
+    - If `--spcomp-dir` isn't specified, the script will try to detect the compiler based on an
+    existing `spcomp` executable in your path.
+    - Do not add `build.ninja` to version control; it should always be generated from
+    `configure.py` as it contains paths specific to your filesystem.  (It's ignored by default,
+    but mentioned here for emphasis.)
+3. Run `ninja`; this will read the `build.ninja` script and build things as necessary.  Files
+will be generated and copied to `build/`, creating any intermediate folders if they don't exist.
+Re-run `ninja` whenever you make changes to rebuild any files as necessary.
+    - You do not have to re-run `python3 configure.py` yourself; running `ninja` will do this
+    for you if `configure.py` itself is modified, and it will pass in the same parameters you
+    originally used.  You may want to re-run it yourself if you change the options.
+    - Any files removed from `configure.py` will remain in `build/`; run `ninja -t cleandead`
+    to remove any lingering outputs.
+    - In case you need to wipe the build outputs, delete `build.ninja` and `build/`, then start
+    from step 2.

+ 125 - 0
configure.py

@@ -0,0 +1,125 @@
+#!/usr/bin/python
+
+# plugin names, relative to `scripting/`
+plugins = [
+	'simple-chatprocessor.sp',
+]
+
+# files to copy to builddir, relative to root
+# plugin names from previous list will be copied automatically
+copy_files = [ ]
+
+# additional directories for sourcepawn include lookup
+# `scripting/include` is explicitly included
+include_dirs = [
+	# 'third_party/vendored'
+]
+
+# required version of spcomp (presumably pinned to SM version)
+spcomp_min_version = (1, 9)
+
+########################
+# build.ninja script generation below.
+
+import contextlib
+import misc.ninja_syntax as ninja_syntax
+import misc.spcomp_util
+import os
+import sys
+import argparse
+import platform
+import shlex
+import shutil
+import subprocess
+
+parser = argparse.ArgumentParser('Configures the project.')
+parser.add_argument('--spcomp-dir',
+		help = 'Directory with the SourcePawn compiler.  Will check PATH if not specified.')
+
+args = parser.parse_args()
+
+print("""Checking for SourcePawn compiler...""")
+spcomp = shutil.which('spcomp', path = args.spcomp_dir)
+if 'x86_64' in platform.machine():
+	# Use 64-bit spcomp if architecture supports it
+	spcomp = shutil.which('spcomp64', path = args.spcomp_dir) or spcomp
+if not spcomp:
+	raise FileNotFoundError('Could not find SourcePawn compiler.')
+
+available_version = misc.spcomp_util.extract_version(spcomp)
+version_string = '.'.join(map(str, available_version))
+print('Found SourcePawn compiler version', version_string, 'at', os.path.abspath(spcomp))
+
+if spcomp_min_version > available_version:
+	raise ValueError("Failed to meet required compiler version "
+			+ '.'.join(map(str, spcomp_min_version)))
+
+# properly handle quoting within params
+if platform.system() == "Windows":
+	arg_list = subprocess.list2cmdline
+else:
+	arg_list = shlex.join
+
+with contextlib.closing(ninja_syntax.Writer(open('build.ninja', 'wt'))) as build:
+	build.comment('This file is used to build SourceMod plugins with ninja.')
+	build.comment('The file is automatically generated by configure.py')
+	build.newline()
+	
+	vars = {
+		'configure_args': arg_list(sys.argv[1:]),
+		'root': '.',
+		'builddir': 'build',
+		'spcomp': spcomp,
+		'spcflags': [ '-i${root}/scripting/include', '-h', '-v0' ]
+	}
+	
+	vars['spcflags'] += ('-i{}'.format(d) for d in include_dirs)
+	
+	for key, value in vars.items():
+		build.variable(key, value)
+	build.newline()
+	
+	build.comment("""Regenerate build files if build script changes.""")
+	build.rule('configure',
+			command = sys.executable + ' ${root}/configure.py ${configure_args}',
+			description = 'Reconfiguring build', generator = 1)
+	
+	build.build('build.ninja', 'configure',
+			implicit = [ '${root}/configure.py', '${root}/misc/ninja_syntax.py' ])
+	build.newline()
+	
+	build.rule('spcomp', deps = 'msvc',
+			command = '"${spcomp}" ${in} ${spcflags} -o ${out}',
+			description = 'Compiling ${out}')
+	build.newline()
+	
+	# Platform-specific copy instructions
+	if platform.system() == "Windows":
+		build.rule('copy', command = 'cmd /c copy ${in} ${out} > NUL',
+				description = 'Copying ${out}')
+	elif platform.system() == "Linux":
+		build.rule('copy', command = 'cp ${in} ${out}', description = 'Copying ${out}')
+	build.newline()
+	
+	build.comment("""Compile plugins specified in `plugins` list""")
+	for plugin in plugins:
+		smx_plugin = os.path.splitext(plugin)[0] + '.smx'
+		
+		sp_file = os.path.normpath(os.path.join('$root', 'scripting', plugin))
+		
+		smx_file = os.path.normpath(os.path.join('$builddir', 'plugins', smx_plugin))
+		build.build(smx_file, 'spcomp', sp_file)
+	build.newline()
+	
+	build.comment("""Copy plugin sources to build output""")
+	for plugin in plugins:
+		sp_file = os.path.normpath(os.path.join('$root', 'scripting', plugin))
+		
+		dist_sp = os.path.normpath(os.path.join('$builddir', 'scripting', plugin))
+		build.build(dist_sp, 'copy', sp_file)
+	build.newline()
+	
+	build.comment("""Copy other files from source tree""")
+	for filepath in copy_files:
+		build.build(os.path.normpath(os.path.join('$builddir', filepath)), 'copy',
+				os.path.normpath(os.path.join('$root', filepath)))

+ 183 - 0
misc/ninja_syntax.py

@@ -0,0 +1,183 @@
+#!/usr/bin/python
+
+"""Python module for generating .ninja files.
+
+Note that this is emphatically not a required piece of Ninja; it's
+just a helpful utility for build-file-generation systems that already
+use Python.
+"""
+
+import re
+import textwrap
+
+def escape_path(word):
+    return word.replace('$ ', '$$ ').replace(' ', '$ ').replace(':', '$:')
+
+class Writer(object):
+    def __init__(self, output, width=78):
+        self.output = output
+        self.width = width
+
+    def newline(self):
+        self.output.write('\n')
+
+    def comment(self, text):
+        for line in textwrap.wrap(text, self.width - 2, break_long_words=False,
+                                  break_on_hyphens=False):
+            self.output.write('# ' + line + '\n')
+
+    def variable(self, key, value, indent=0):
+        if value is None:
+            return
+        if isinstance(value, list):
+            value = ' '.join(filter(None, value))  # Filter out empty strings.
+        self._line('%s = %s' % (key, value), indent)
+
+    def pool(self, name, depth):
+        self._line('pool %s' % name)
+        self.variable('depth', depth, indent=1)
+
+    def rule(self, name, command, description=None, depfile=None,
+             generator=False, pool=None, restat=False, rspfile=None,
+             rspfile_content=None, deps=None):
+        self._line('rule %s' % name)
+        self.variable('command', command, indent=1)
+        if description:
+            self.variable('description', description, indent=1)
+        if depfile:
+            self.variable('depfile', depfile, indent=1)
+        if generator:
+            self.variable('generator', '1', indent=1)
+        if pool:
+            self.variable('pool', pool, indent=1)
+        if restat:
+            self.variable('restat', '1', indent=1)
+        if rspfile:
+            self.variable('rspfile', rspfile, indent=1)
+        if rspfile_content:
+            self.variable('rspfile_content', rspfile_content, indent=1)
+        if deps:
+            self.variable('deps', deps, indent=1)
+
+    def build(self, outputs, rule, inputs=None, implicit=None, order_only=None,
+              variables=None, implicit_outputs=None, pool=None):
+        outputs = as_list(outputs)
+        out_outputs = [escape_path(x) for x in outputs]
+        all_inputs = [escape_path(x) for x in as_list(inputs)]
+
+        if implicit:
+            implicit = [escape_path(x) for x in as_list(implicit)]
+            all_inputs.append('|')
+            all_inputs.extend(implicit)
+        if order_only:
+            order_only = [escape_path(x) for x in as_list(order_only)]
+            all_inputs.append('||')
+            all_inputs.extend(order_only)
+        if implicit_outputs:
+            implicit_outputs = [escape_path(x)
+                                for x in as_list(implicit_outputs)]
+            out_outputs.append('|')
+            out_outputs.extend(implicit_outputs)
+
+        self._line('build %s: %s' % (' '.join(out_outputs),
+                                     ' '.join([rule] + all_inputs)))
+        if pool is not None:
+            self._line('  pool = %s' % pool)
+
+        if variables:
+            if isinstance(variables, dict):
+                iterator = iter(variables.items())
+            else:
+                iterator = iter(variables)
+
+            for key, val in iterator:
+                self.variable(key, val, indent=1)
+
+        return outputs
+
+    def include(self, path):
+        self._line('include %s' % path)
+
+    def subninja(self, path):
+        self._line('subninja %s' % path)
+
+    def default(self, paths):
+        self._line('default %s' % ' '.join(as_list(paths)))
+
+    def _count_dollars_before_index(self, s, i):
+        """Returns the number of '$' characters right in front of s[i]."""
+        dollar_count = 0
+        dollar_index = i - 1
+        while dollar_index > 0 and s[dollar_index] == '$':
+            dollar_count += 1
+            dollar_index -= 1
+        return dollar_count
+
+    def _line(self, text, indent=0):
+        """Write 'text' word-wrapped at self.width characters."""
+        leading_space = '  ' * indent
+        while len(leading_space) + len(text) > self.width:
+            # The text is too wide; wrap if possible.
+
+            # Find the rightmost space that would obey our width constraint and
+            # that's not an escaped space.
+            available_space = self.width - len(leading_space) - len(' $')
+            space = available_space
+            while True:
+                space = text.rfind(' ', 0, space)
+                if (space < 0 or
+                    self._count_dollars_before_index(text, space) % 2 == 0):
+                    break
+
+            if space < 0:
+                # No such space; just use the first unescaped space we can find.
+                space = available_space - 1
+                while True:
+                    space = text.find(' ', space + 1)
+                    if (space < 0 or
+                        self._count_dollars_before_index(text, space) % 2 == 0):
+                        break
+            if space < 0:
+                # Give up on breaking.
+                break
+
+            self.output.write(leading_space + text[0:space] + ' $\n')
+            text = text[space+1:]
+
+            # Subsequent lines are continuations, so indent them.
+            leading_space = '  ' * (indent+2)
+
+        self.output.write(leading_space + text + '\n')
+
+    def close(self):
+        self.output.close()
+
+
+def as_list(input):
+    if input is None:
+        return []
+    if isinstance(input, list):
+        return input
+    return [input]
+
+
+def escape(string):
+    """Escape a string such that it can be embedded into a Ninja file without
+    further interpretation."""
+    assert '\n' not in string, 'Ninja syntax does not allow newlines'
+    # We only have one special metacharacter: '$'.
+    return string.replace('$', '$$')
+
+
+def expand(string, vars, local_vars={}):
+    """Expand a string containing $vars as Ninja would.
+
+    Note: doesn't handle the full Ninja variable syntax, but it's enough
+    to make configure.py's use of it work.
+    """
+    def exp(m):
+        var = m.group(1)
+        if var == '$':
+            return '$'
+        return local_vars.get(var, vars.get(var, ''))
+    return re.sub(r'\$(\$|\w*)', exp, string)

+ 16 - 0
misc/spcomp_util.py

@@ -0,0 +1,16 @@
+#!/usr/bin/python3
+
+import subprocess
+import io
+
+def extract_version(spcomp):
+	"""
+	Extract version string from caption in SourcePawn compiler into a tuple.
+	The string is hardcoded in `setcaption(void)` in `sourcepawn/compiler/parser.cpp`
+	"""
+	p = subprocess.Popen([spcomp], stdout=subprocess.PIPE)
+	caption = io.TextIOWrapper(p.stdout, encoding="utf-8").readline()
+	
+	# extracts last element from output in format "SourcePawn Compiler major.minor.rev.patch"
+	*_, version = caption.split()
+	return tuple(map(int, version.split('.')))