The journey to Git-based ABAP development: Part 2

[MarcelloUrbani]

I recently posted a blog about the start of our investigations into Git-based ABAP development here at Basis Technologies. A month or so that has passed since that original piece, so in this Part 2 I want to share the progress we’ve made and the latest lessons learned. 

Let’s start with some context. How do we develop our ABAP-based products today (without Git)? Currently we employ a fairly traditional transport-based workflow in a landscape configuration which is probably familiar to most SAP users:



Fixes are merged from our ‘production’ system into our project dev box on a fairly regular basis (we use our own tool to automate transport management, but it still follows this scheme). 

Project work, on the other hand, is merged into the maintenance development box when testing is completed and a project ‘release’ is ready.

 

As long as you have the means to detect conflicts between incoming (ie. project) changes and those made locally (i.e. in the maintenance dev system) this works well enough. It’s a pretty standard approach, after all. But although this setup exists for many good reasons, there are a number of things that actually make it less than ideal:

• Several developments share the same code base
  
  
  
  • Sometimes they have opaque dependencies which can cause import errors downstream
  
  
  • Sometimes independent requirements affect the same piece of code
  
  
  • Rejected changes accumulate in the dev systems over time
  
  
  
  


• We need to maintain a separate server (or more) for each version we need to regularly patch

 

Given these limitations - which I know will be familiar to just about anyone developing in ABAP systems - we have been looking to move forward for a while, and a big upcoming project made it urgent.

So what else could we do? Well the rest of the world (ie. not in SAP) is mostly working something like this:



where the CI server creates builds and runs automated testers before handing them to testers and/or production systems. And they’re probably using something like gitflow to manage releases.

 

AbapGit does give us the means to do distributed development in SAP (well, as long as you can give each developer an independent server), but for several reasons I don’t think it’s enough to implement the same approach:

1. We don’t have a ‘build’ as such - just a stream of changes


2. Only TMS is currently reliable enough to deliver to production


3. We need to transport customising too (very little in our case, but in many places it’s a huge issue)


4. We have a sizable WIP in our TMS landscape

 

That being the case, perhaps something like this is the closest we can get in a SAP environment:



But this would be such a big change we decided it’s too ambitious to try it all in one go. Better to start with some baby steps. So we decided to go for a hybrid solution, where development done in Git (via abapGit) generates a transport which is then merged into our TMS landscape:



What follows is an account of my first attempt at this workflow. We plan to have a few more iterations with our current landscape before starting the big project, which we would rather start with systems that are much better aligned (the TMS dev box has a lot of WIP/abandoned changes).

First quirk

After I did my initial development on my personal system, abapGit thought I also added an include to another function group:



Which I didn’t.

It’s not a big deal, I could simply ignore it while committing, but would be much harder if I worked on dozens of includes rather than one, and anyway I want this to run as smoothly as possible.

 

First fix attempt

I didn’t touch that object and didn’t want to see it as changed for some glitch in the system. So to fix the problem I had the silly idea of updating abapGit to the latest version in my personal dev system, which ended up showing me lots of extra changes due to changes in its serialization algorithms (i.e. a flag added to an object type looks like a change to every object of that type).

 

Luckily Git is very good at fixing this sort of thing. I updated the central Git dev box, committed all the phantom changes, and was back with my single extra function group change.

Second fix attempt

So now I’m on the latest version of abapGit but still have my problem. Next I tried to regenerate the function group index in SE80. Didn’t work, and in the debugger I found out that rather than a bug in abapGit it was a bad entry in a table. Regardless, the outcome was that on 2 identical systems cloned from the same parent, the include was found on one and not the other.

I gave up and committed only the relevant change, ignoring the function group.

Oh well, I guess I’ll refresh my dev box from a backup, which usually takes about a minute of my time and 15 of waiting (these servers are cattle, not pets)

Second quirk

When I synced the main Git dev system with my branch (check out the gitflow article I mentioned earlier to get a better idea of what branches are), abapGit created a transport with the whole function group, even if the change was only on a single include.

 

A function group is safer to export in a different system, but also increases the chances of conflicts when using multiple tracks, which is exactly why I need that transport, so I replaced it with a TOC with only the include I changed.

Don’t think this is an option for big changes and continuous integration though.

 

Conflict resolution

Even the single include I did change had a conflict with a local change in the TMS dev box. This screenshot shows what happened when I analysed the transport containing my change before it was imported from the central Git dev to TMS dev:



I could fix this manually as I’ve always had to in the past, but decided to exploit the power of Git instead.

So I imported the offending include the other way around, from the TMS dev box to my personal git dev, and committed this change on the current release branch (should have used another one, but I decided to cut a corner for once).



 

Then I went to my favourite text editor to check the situation.



...and rebased the change. That took me here:



With hindsight, a merge would have been more appropriate in this case, as I had to force a push. 

 

One should never force a push in a shared git repository as it makes it difficult for others to sync their personal repos. On a release branch like this is ok, but still a bad habit. We will probably forbid it when we go live with this.

 

The blame annotations (the left part of the picture, which for every line of code tells you who last changed it, when and why ) show that both changes are in the final source code:



Then I synced the central Git dev box with this revision and imported the resulting transport into the main TMS dev box. From here will move forward with good old TMS.

It showed the same conflict, but I ignored it knowing the conflict reported was already merged this time.

Finally I advanced the currentRelease branch to this version (master is used for other purposes). This was possible because there was no concurrent change, but a proper workflow should address this better, maybe merging before creating the transport. Or probably after releasing it - we haven’t decided yet.

Lessons learned

As you can see, we’ve definitely moved a few steps forward but we have learned a few things on the way (which is really the point of this experimentation anyway). For example:

• Always make sure to use the same version of abapGit in every relevant system. Perhaps sync it with a branch of your own repository. Keeping it up to date is usually a good idea, but you really want to decide yourself when to do it! And warn everyone before you do.


• Always check the state of the repository before you start coding, even if you’re the only guy in the system.

 

That’s Part 2 on our progress towards an abapGit workflow. There’s still plenty to figure out so I’m going to keep posting as we move forward. Watch out for Part 3 in the series fairly soon.