Miosix and git workflow

The Miosix git repository

Miosix uses the git version control system to manage the kernel source code. To download and use the kernel you have to clone its git repository. The main Miosix git repository is hosted on this website, miosix.org, but we have no web page where you can browse the kernel sources. For this, we use a github mirror and the two repositories will be always kept in sync.

As the kernel is under active development, users are advised to periodically git fetch or check the kernel page on github to look for new features and bug fixes, and pull the changes if required.

By default, when cloning a repository, git will download the master branch. Miosix offers two additional branches, testing and unstable, and you should be aware of the conventions we use for how we manage branches:

• The master branch contains the stable kernel, it is updated infrequently, usually once a year when a new release is made. Sometimes, a few minor releases are updated in short order.
• The testing branch contains the in-development code that we consider almost ready. Use this branch to get more bleeding edge features.
• The unstable branch is where the Miosix development team is working. It's meant to be used only by people in contact with the development team, as we occasionally rebase and force-push changes, so you may have a hard time keeping your git repository in sync with ours (the commit you downloaded today may no longer exist after a force push). If you're writing applications, it's best to ignore this branch and wait for the desired change to trickle to testing.

Cloning the kernel repository from miosix.org can be done with:

git clone https://miosix.org/git-public/miosix-kernel.git

While cloning the github mirror with:

git clone https://github.com/fedetft/miosix-kernel.git

To checkout a specific branch, once you have cloned the git repository, use the following command

cd miosix-kernel
git checkout testing

The rest of this page explains how to set up a software project making use of the Miosix kernel, that can later allow updating the kernel through git without creating problems. There are basically two ways to do so:

• Setting up out an out of git tree project: this is the suggested workflow for developing applications using Miosix. Your application directory and git repo stay separate from the kernel directory and git repo. The kernel can be a git submodule of your project.
• Forking the Miosix kernel git repository. This is the suggested workflow for extending the kernel, such as adding new board support packages (BSPs).

The two workflows can be combined, for example you may want to fork the kernel to add your BSPs, and then use the forked kernel as submodule for developing your applications.

Setting up an out of git tree project

Out of git tree projects allow you to write applications in a directory that stays separate from the Miosix kernel directory. Additionally, they allow to perform the kernel configuration (that requires editing configuration files) without the need to make changes to files within the Miosix git repo at all. This last point is important both to have the Miosix git repository as a git submodule of your application's git repository, and to allow sharing the same downloaded copy of the Miosix kernel across different applications, each with its own configuration. The kernel source code size on the hard drive has grown bigger in recent years due to the inclusion of headers with peripheral registers definitions in the CMSIS directory, so you may want to share a single downloaded copy of the kernel across several projects.

Note that the ability to develop applications without making changes to the kernel directory is currently limited to the use case of configuring the kernel for one of the available BSPs. If you need to add BSPs to the kernel, then you'll most likely also have to fork the kernel repository. The most likely is due to generic BSPs in the kernel, which can be identified in the list of boards in Makefile.inc as <chip name>_generic such as stm32f103xb_generic. If you're writing an application for a board that has a chip with a generic BSP in the kernel, you can (unless you really need to run some custom code during boot) get away with selecting the generic BSP without forking the kernel. If there's no generic BSP for your chip, or need to add support for an entirely new chip, then you'll have to fork the kernel to add it.

Suppose you have a git repository called test, which is the repo of your application, and you would like to download the miosix-kernel repository as a git submodule. Then open a shell in the test repository and type

git submodule init
git submodule add https://miosix.org/git-public/miosix-kernel.git miosix-kernel
cd miosix-kernel
git fetch origin
cd ..
git commit -a -m "Added miosix-kernel as a git submodule"

This solves the problem from the git side, now let's solve the problem from the Miosix side. As you may know, git submodules work well if you never make changes in the submodule directory, otherwise you will have to basically fork the submodule, or you will find yourself making changes to a repository in a detached head state and risk losing them at the next git pull.

However, the Makefile for Miosix, as well as configuration files such as miosix/config/Makefile.inc and miosix/config/miosix_settings.h are within the submodule directory, so how are you going to make use of the kernel without touching files in the submodule directory?

The solution is out of git tree projects, a new feature added in Miosix 2.0 that basically moves the main.cpp, Makefile, config directory, and the Netbeans IDE project directory outside of the git repository, using tweaks to the Makefile paths so that the kernel will find the config directory outside of the kernel, not the one inside it.

To get started with out of git tree projects, there is a perl script that is used to easily set everything up. For example, suppose you are in the directory of your git repository where you have added the miosix-kernel submodule, and want to set up an out of git tree project there, open a shell and type

perl miosix-kernel/miosix/_tools/init_project_out_of_git_tree.pl

The script will create a Miosix out of git tree project in the directory it is called, basically creating a config directory with a copy of the configuration files you can edit, a Makefile and a main.cpp as usual, as well as a new miosix_np_2 directory with a Netbeans IDE project.

The main advantage of this solution is that it keeps your application and the kernel in two separate git repositories. The disadvantage is that if you need to make changes to the kernel other than to configuration files, you will need to clone the miosix-kernel repository anyway, as well as having to manage the complexity of having two git repositories.

Forking the Miosix git repository

Forking the Miosix git repository simply means cloning the kernel repository and start making changes within the kernel. To clone the git repository you simply type the following commands:

git clone https://miosix.org/git-public/miosix-kernel.git
cd miosix-kernel
git checkout testing

The Miosix top-level directory contains the following files and subdirectories:

The README.md file contains more detailed information on how the kernel source code directory tree is structured, but a quick overview is that the entire kernel code is in the miosix subdirectory, templates contains application templates you can copy out of the kernel directory to create your out of tree projects, examples contains some demos of the Miosix APIs, wile tools contains scripts and support code that does not get compiled into the kernel. It also contains the scripts and patches used to build the Miosix Toolchain, the GCC-based cross compiler used to build the kernel.

Making changes

Once you start making changes you can simply commit in the cloned git repository. If you need to share your changes, perhaps because there are multiple developers, git allows to have multiple remotes for a repository, so you can leave the default remote to periodically pull updates and bugfixes from the upstream Miosix repository, as well as push and pull from your personal remote repository.

Testing changes

Once you start making changes to the kernel, you'll most likely want to quickly test to see if they work, which entails compiling the kernel to check for compile-time errors, and flash it to a board to check the code works at run-time.

To do so you can set up a Miosix out of git tree project as described earlier that points to your cloned kernel, but there's a quicker way: building the templates from within the kernel tree.

To do so, you'll need to select the board you want to build directly in the miosix/config/Makefile.inc inside the kernel source tree, and select additional options (if needed) directly in the miosix/config/miosix_settings.h inside the kernel source tree.

Ad this point, you can cd to the templates/simple (or any other template directory) and type

make
make program

from that directory.

Although convenient, this comes with two caveats that you should be careful about:

• If you also use the kernel for making out of tree projects, the scripts such as init_project_out_of_git_tree.pl, when copying the configuration and template directory to initialize a out of tree project, will also copy the testing code with it. This is often just a minor inconvenience, which can easily be solved in many ways, such as by using git stash before running an init_project_out_of_git_tree.pl script to temporarily stash those changes.
• You should remember to not commit these changes to configuration files, especially when you plan to contribute patches upstream. The list of boards in Makefile.inc should by default have all the available boards commented out, so as to let users select the board they want by uncommenting it.

Running the Miosix regression testsuite

Miosix comes with a regression test suite that exercises most of the kernel code and is useful when making changes to the kernel to verify the absence of regressions.

To compile the testsuite you should follow the instructions above to moddify the kernel's miosix/config directory to select the board you want to run the testsuite on, and then compile the testuite that is in the tools/testsuite directory.

cd tools/testsuite
make
make program

Note that compiled firmware of the testsuite is quite large since it contains every function of the kernel, especially if compiled with support for the filesystem and userspace processes.

Making it easier to run the testsuite on small microcontrollers with little FLASH and RAM memory is a work in progress, he Miosix development team "chops" the testsuite in smaller pieces by commenting out part of it and running it one piece at a time, this is a tip worth knowing if you need to run it on very small microcontrollers.

Contributing to Miosix

Miosix uses a shared ownership model, meaning that you retain the copyright for the contributions you add to the kernel. Your contributions should however be licensed with the same license of the rest of the kernel:

/***************************************************************************
 *   Copyright (C) <year> by <your name>                                   *
 *                                                                         *
 *   This program is free software; you can redistribute it and/or modify  *
 *   it under the terms of the GNU General Public License as published by  *
 *   the Free Software Foundation; either version 2 of the License, or     *
 *   (at your option) any later version.                                   *
 *                                                                         *
 *   This program is distributed in the hope that it will be useful,       *
 *   but WITHOUT ANY WARRANTY; without even the implied warranty of        *
 *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the         *
 *   GNU General Public License for more details.                          *
 *                                                                         *
 *   As a special exception, if other files instantiate templates or use   *
 *   macros or inline functions from this file, or you compile this file   *
 *   and link it with other works to produce a work based on this file,    *
 *   this file does not by itself cause the resulting work to be covered   *
 *   by the GNU General Public License. However the source code for this   *
 *   file must still be made available in accordance with the GNU General  *
 *   Public License. This exception does not invalidate any other reasons  *
 *   why a work based on this file might be covered by the GNU General     *
 *   Public License.                                                       *
 *                                                                         *
 *   You should have received a copy of the GNU General Public License     *
 *   along with this program; if not, see <http://www.gnu.org/licenses/>   *
 ***************************************************************************/

Be sure to add the copyright notice to all nontrivial source files. An example of a trivial source file would be a forwarding header such as this one.

If you wrote the file you want to add from scratch, just add your name to the copyright notice. If instead you copy-pasted the file from another one already present in the kernel and modified it (this often happens when making BSPs as there's almost always a similar BSP in the kernel to take inspiration from), just append your name to the list of authors of the file you copied.

Since we use github only as a mirror to our repository that we host on miosix.org, we do not accept pull requests. You will instead be required to make a format-patch with the changes you want to be pushed upstream and send it via email to fede.tft&&miosix.org (s/&&/@/). It may be a good idea to get in contact first to discuss the changes you want pushed upstream, especially if they include modifications to the kernel itself rather than being simply new BSPs.

When submitting patches for inclusion, please make sure that commits are small and self-contained, in order to simplify the integration process.

Your changes should apply to the latest testing branch. If changes are related to the kernel itself, we may request to rebase them on top of unstable.