Reproducibility
by any means necessary
Lessons from MATLAB

Dr David Wilby
Research Software Engineer
British Antarctic Survey

Wed 20th November 2024

Reproducibility

Let’s start by assuming that, in this audience at least, we can agree that open research practices are important.

When it comes to sharing our code openly, that’s a good start, but it would be even better if the code worked for other people.

So they can use it, examine what we’ve done and build upon it.

When it comes to code, this is easier said than done.

Some elements of reproducible code

0. Availability
1. FAIR data
2. Code readability
3. Documentation
4. Follow community conventions
5. Dependency specification
6. Try your best!

0. Availability - code and any required software must be attainable
1. FAIR data -
2. Code readability - technically not absolutely necessary
3. Documentation - most code is not implicitly obvious how to use
4. Follow community conventions - it helps if people find code laid out how they expect, don’t reinvent the wheel
5. Dependency specification - most code depends on other code or software, you should define how to get it and which version
6. Try your best! - it’s actually kind of difficult and every case is different, don’t give up!

MATLAB

• Mathematical programming language turned general purpose language
• Relatively user-friendly,
• Bundled integrated development environment (IDE),
• Historically popular in many undergrad courses, particularly engineering,
• Has some built-in toolboxes and functionality for maths/modelling,
• Proprietary - commercial license
• Every Russell Group University has a MATLAB license
• Has relatively good documentation

https://en.m.wikipedia.org/wiki/File:Matlab_Logo.png

Proprietary software and reproducibility

• Arguably/Genuinely obtaining a license for MATLAB is a barrier to reproducibility
• Code and data formats may require paid for software to read
• It is harder for the community to create their own tooling, relying on the owners to enable it
• So why bother?
• My position is that it is better to pursue reproducibility than to just give up.

The missing package manager

One notable missing tool is one to install dependencies. In software, we nearly always use other people’s software to make our software work. Software is sometimes organised as “packages” - installable, versioned pieces of software. In other languages we use “package managers”.

Possible solutions…

MATLAB weirdly released something called a ‘package manager’ earlier this year

Possible solutions:

• Just document the dependencies,
• Actually copy in the dependencies’ code (if reasonable)

So what do we do about it?!

• Try your best!
• Apply an open source license to your code
• Make the most open choices you can
• Find (or create) a community
• Advocate for better
• Maybe choose a different tool next time?

• If you don’t try, your code won’t be reproducible
• Make you code available on GitHub, GitLab, Zenodo or similar AND give it an open source license so others can reuse it
• Choose open data and file formats wherever possible - in MATLAB avoid .mat and .fig files, maybe avoid livescripts but consider creating a repo that can be opened in MATLAB online
• Work with your colleagues or friends to try out your code and point out where more might be needed to make it reproducible, reach out to experts at your institution or in your field
• Ask MathWorks/the owners of your software to consider improving facilities for reproducibility in the language
• Think about whether the tool you’re used to using is really serving your purposes

Clearing up a misconception

Your code can be open source
even if the language itself is proprietary

However you must give it a license

It’s not all bad!

• Python and Jupyter notebook integration
• Github actions
• Matlab Online
  • e.g. Open repo in MATLAB online

• MathWorks are working hard to integrate with open source

Lessons

• Make good choices
• Work with a community
• Make your code available
• Try your best!

Plug!

Come and contribute to the reproducible MATLAB project!

github.com/reproducibleMATLAB

Image Credits 📷

On Unsplash:

• Joshua Sortino
• National Cancer Institute
• Thomas Kinto
• Jr Korpa
• Stormseeker
• Batu Gezer
• Travis Murdock
• Thom Milkovic
• Pawel Czerwinski
• NASA