Miosix and git workflow

The Miosix git repository

Miosix uses the git version control system to manage the kernel source code. To download and use the kernel you have to clone its git repository. The main Miosix git repository is hosted on this website, miosix.org, but we have no web page where you can browse the kernel sources. For this, we use a github mirror and the two repositories will be always kept in sync.

As the kernel is under active development, users are advised to periodically git fetch or check the kernel page on github to look for new features and bug fixes, and pull the changes if required.

Cloning the kernel repository from miosix.org can be done with:

git clone https://miosix.org/git-public/miosix-kernel.git

While cloning the github mirror with:

git clone https://github.com/fedetft/miosix-kernel.git

Branches

By default, when cloning a repository, git will download the master branch. Miosix offers two additional branches, testing and unstable, and you should be aware of the conventions we use for how we manage branches:

• The master branch contains the stable kernel, it is updated infrequently, usually once a year when a new release is made. Sometimes, a few minor releases are updated in short order.
• The testing branch contains the in-development code that we consider almost ready. Use this branch to get more bleeding edge features.
• The unstable branch is where the Miosix development team is working. It's meant to be used only by people in contact with the development team, as we occasionally rebase and force-push changes, so you may have a hard time keeping your git repository in sync with ours (the commit you downloaded today may no longer exist after a force push). If you're writing applications, it's best to ignore this branch and wait for the desired change to trickle to testing.

To checkout a specific branch, once you have cloned the git repository, use the following command

cd miosix-kernel
git checkout testing

Using Miosix in your projects

The rest of this page explains how to set up a software project making use of the Miosix kernel. There are basically two ways to do so:

• Using out of git tree projects: this is the suggested workflow for developing applications using Miosix. Your application directory (and optionally git repo) stay separate from the kernel directory and git repo. The kernel can be a git submodule of your project.
• Forking the Miosix kernel git repository. This is the suggested workflow for extending the kernel, such as adding new board support packages (BSPs).

The two workflows can be combined, for example you may want to fork the kernel to add your BSPs, and then use the forked kernel as submodule for developing your applications.

Out of git tree projects

Out of git tree projects allow you to write applications in a directory that stays separate from the Miosix kernel directory. When we say out of git tree, we mean that your project is developed outside of the Miosix git tree. Your project can be a git repository too, but that is up to you to decide.

Additionally, a key part of an out of git tree project is that it allows to perform the kernel configuration (that requires editing configuration files) without the need to make changes to files within the Miosix git repo at all. This last point is important both to have the Miosix git repository as a git submodule of your application's git repository, and to allow sharing the same downloaded copy of the Miosix kernel across different applications, each with its own configuration. The kernel source code size on the hard drive has grown bigger in recent years due to the inclusion of headers with peripheral registers definitions in the CMSIS directory, so you may want to share a single downloaded copy of the kernel across several projects.

Note that the ability to develop applications without making changes to the kernel directory is currently limited to the use case of configuring the kernel for one of the available BSPs. If you need to add BSPs to the kernel, then you'll most likely also have to fork the kernel repository. The most likely is due to generic BSPs in the kernel, which can be identified in the list of boards in Makefile.inc as <chip name>_generic such as stm32f103xb_generic. If you're writing an application for a board that has a chip with a generic BSP in the kernel, you can (unless you really need to run some custom code during boot) get away with selecting the generic BSP without forking the kernel. If there's no generic BSP for your chip, or need to add support for an entirely new chip, then you'll have to fork the kernel to add it.

Organizing your out of git tree projects

For an out of git tree project, you need to create a directory in your computer that will contain your project. The miosix-kernel directory (and git repo) can either be a subdirectory of your project's directory (common option when adding the Miosix kernel as a git submodule to your project):

Or you can organize your projects side by side to the kernel (common when sharing a single kernel among multiple projects):

The only constraint (it's a limitation in the Makefile build systems), is that the relative path from your project directory to the kernel should not contain spaces. This practically means that you should not give your project directories names with spaces.

Adding Miosix as a submodule to your git project

Suppose you have a git repository called myapplication, which is the repo of your application, and you would like to download the miosix-kernel repository as a git submodule. Then open a shell in the test repository and type:

git submodule init
git submodule add https://miosix.org/git-public/miosix-kernel.git miosix-kernel
git commit -m "Added miosix-kernel as a git submodule"

Initializing an out of git tree project

Using out of git tree projects implies having a build system in your project directory that can both compile the source code of your application and the Miosix kernel. Miosix provides two build systems, Makefiles and CMake. In this tutorial we'll use Makefiles.

Miosix projects start from a template that you can find in the miosix-kernel/templates directory. You can copy the template yourself from the kernel source tree to your project directory, but in that case, you'll need to fix the relative paths in the build system. Alternatively, you can use the init_project_out_of_git_tree.pl script to initialize a project.

Additionally, the kernel configuration directory needs to be copied to your project. This is necessary because you'll need to make changes to those files to select kernel build options, at minimum which board you want to compile the kernel for. As you may know, git submodules work well if you never make changes in the submodule directory, otherwise you will have to basically fork the submodule, or you will find yourself making changes to a repository in a detached head state and risk losing them at the next git pull. For this reason, we'll copy the config directory to your project and make changes there. The init_project_out_of_git_tree.pl performs also this action for you.

Initialize a project using init_project_out_of_git_tree.pl

Just open a shell in your project directory and invoke the perl script in the kernel:

perl <path to the kernel>tools/init_project_out_of_git_tree.pl

Thus for the case when the kernel is a subdirectory of your project that would be:

perl miosix-kernel/tools/init_project_out_of_git_tree.pl

While for the side-by-side case:

perl ../miosix-kernel/tools/init_project_out_of_git_tree.pl

Note that you have to run the script without moving the current directory away from the one of your project. This means you can't cd to the miosix-kernel/tools directory, run the script there, and then cd back. The script withes the project files to the current directory it's been invoked from.

If all goes well you'll get a message similar to:

Successfully created Miosix project
Target directory: myapplication
Kernel directory: myapplication/miosix-kernel/miosix

and your project should now contain the config directory and the main.cpp, Makefile and CMakeLists.txt files.

From there, you'll need to edit the configuration as desired, and start building your application starting from main.cpp.

Manually copying a template

The perl script is convienient, but not necessary. It's possible to do the configuration manually by copying and editing files.

To do so, there are 3 steps involved.

First, copy the miosix-kernel/miosix/config directory to your project.

Second, copy a template form the miosix-kernel/templates directory to your project. The simple template configures the kernel as a unikernel and will run your application in kernelspace. The processes template will allow you to develop (part of) your application in a userspace process with memory protection, bu that's a more advanced topic realted to the fliod kernel concept.

You shouldn't copy the template directory, rather its content, so the main.cpp, Makefile and CMakeLists.txt files in the case of the simple template.

Third and final step, you should edit the build system to point to the correct kernel and config directory. This step was done automatically by the perl script, in this case you'll have to do it manually.

Open the Makefile file, and edit the following lines:

KPATH := ../../miosix
CONFPATH := $(KPATH)

The first line, defining the KPATH variable should be assigned to the relative path from your project directory to the kernel directory. If your miosix-kernel is a subdirectory of your project that should be:

KPATH := miosix-kernel/miosix

The second line defines the CONFPATH variable, and since you want to the kernel to find the config directory in your project , and not the one in the miosix-kernel/miosix/config directory, you'll have to chege it to the current directory, or .:

CONFPATH := .

Finally, open CMakeLists.txt. Also in this case you'll have to tell it where is the kernel directory, and to use the config directory in your project. The kernel directory is specified by replacing:

set(MIOSIX_KPATH ${CMAKE_SOURCE_DIR}/../../miosix)

with:

set(MIOSIX_KPATH ${CMAKE_SOURCE_DIR}/miosix-kernel/miosix)

Whiel to tell CMake to use the config directory in your project, uncomment the following line:

set(MIOSIX_USER_CONFIG_PATH ${CMAKE_SOURCE_DIR}/config)

Forking the Miosix git repository

Forking the Miosix git repository simply means cloning the kernel repository and start making changes within the kernel. To clone the git repository you simply type the following commands:

git clone https://miosix.org/git-public/miosix-kernel.git
cd miosix-kernel
git checkout testing

The Miosix top-level directory contains the following files and subdirectories:

The README.md file contains more detailed information on how the kernel source code directory tree is structured, but a quick overview is that the entire kernel code is in the miosix subdirectory, templates contains application templates you can copy out of the kernel directory to create your out of tree projects, examples contains some demos of the Miosix APIs, wile tools contains scripts and support code that does not get compiled into the kernel. It also contains the scripts and patches used to build the Miosix Toolchain, the GCC-based cross compiler used to build the kernel.

Making changes

Once you start making changes you can simply commit in the cloned git repository. If you need to share your changes, perhaps because there are multiple developers, git allows to have multiple remotes for a repository, so you can leave the default remote to periodically pull updates and bugfixes from the upstream Miosix repository, as well as push and pull from your personal remote repository.

Testing changes

Once you start making changes to the kernel, you'll most likely want to quickly test to see if they work, which entails compiling the kernel to check for compile-time errors, and flash it to a board to check the code works at run-time.

To do so you can set up a Miosix out of git tree project as described earlier that points to your cloned kernel, but there's a quicker way: building the templates from within the kernel tree.

To do so, you'll need to select the board you want to build directly in the miosix/config/Makefile.inc inside the kernel source tree, and select additional options (if needed) directly in the miosix/config/miosix_settings.h inside the kernel source tree.

Ad this point, you can cd to the templates/simple (or any other template directory) and type

make
make program

from that directory.

Although convenient, this comes with two caveats that you should be careful about:

• If you also use the kernel for making out of tree projects, the scripts such as init_project_out_of_git_tree.pl, when copying the configuration and template directory to initialize a out of tree project, will also copy the testing code with it. This is often just a minor inconvenience, which can easily be solved in many ways, such as by using git stash before running an init_project_out_of_git_tree.pl script to temporarily stash those changes.
• You should remember to not commit these changes to configuration files, especially when you plan to contribute patches upstream. The list of boards in Makefile.inc should by default have all the available boards commented out, so as to let users select the board they want by uncommenting it.

Running the Miosix regression testsuite

Miosix comes with a regression test suite that exercises most of the kernel code and is useful when making changes to the kernel to verify the absence of regressions.

To compile the testsuite you should follow the instructions above to moddify the kernel's miosix/config directory to select the board you want to run the testsuite on, and then compile the testuite that is in the tools/testsuite directory.

cd tools/testsuite
make
make program

Note that compiled firmware of the testsuite is quite large since it contains every function of the kernel, especially if compiled with support for the filesystem and userspace processes.

Making it easier to run the testsuite on small microcontrollers with little FLASH and RAM memory is a work in progress, he Miosix development team "chops" the testsuite in smaller pieces by commenting out part of it and running it one piece at a time, this is a tip worth knowing if you need to run it on very small microcontrollers.

Contributing to Miosix

Miosix uses a shared ownership model, meaning that you retain the copyright for the contributions you add to the kernel. Your contributions should however be licensed with the same license of the rest of the kernel:

/***************************************************************************
 *   Copyright (C) <year> by <your name>                                   *
 *                                                                         *
 *   This program is free software; you can redistribute it and/or modify  *
 *   it under the terms of the GNU General Public License as published by  *
 *   the Free Software Foundation; either version 2 of the License, or     *
 *   (at your option) any later version.                                   *
 *                                                                         *
 *   This program is distributed in the hope that it will be useful,       *
 *   but WITHOUT ANY WARRANTY; without even the implied warranty of        *
 *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the         *
 *   GNU General Public License for more details.                          *
 *                                                                         *
 *   As a special exception, if other files instantiate templates or use   *
 *   macros or inline functions from this file, or you compile this file   *
 *   and link it with other works to produce a work based on this file,    *
 *   this file does not by itself cause the resulting work to be covered   *
 *   by the GNU General Public License. However the source code for this   *
 *   file must still be made available in accordance with the GNU General  *
 *   Public License. This exception does not invalidate any other reasons  *
 *   why a work based on this file might be covered by the GNU General     *
 *   Public License.                                                       *
 *                                                                         *
 *   You should have received a copy of the GNU General Public License     *
 *   along with this program; if not, see <http://www.gnu.org/licenses/>   *
 ***************************************************************************/

Be sure to add the copyright notice to all nontrivial source files. An example of a trivial source file would be a forwarding header such as this one.

If you wrote the file you want to add from scratch, just add your name to the copyright notice. If instead you copy-pasted the file from another one already present in the kernel and modified it (this often happens when making BSPs as there's almost always a similar BSP in the kernel to take inspiration from), just append your name to the list of authors of the file you copied.

Since we use github only as a mirror to our repository that we host on miosix.org, we do not accept pull requests. You will instead be required to make a format-patch with the changes you want to be pushed upstream and send it via email to fede.tft&&miosix.org (s/&&/@/). It may be a good idea to get in contact first to discuss the changes you want pushed upstream, especially if they include modifications to the kernel itself rather than being simply new BSPs.

When submitting patches for inclusion, please make sure that commits are small and self-contained, in order to simplify the integration process.

Your changes should apply to the latest testing branch. If changes are related to the kernel itself, we may request to rebase them on top of unstable.