Skip to main content
Detective Siluete

The mystery of R3trans

A few weeks ago I was applying a SPS on a SLT system using Software Update Manager. Sometimes when using the SUM you find some troubles right in the middle of the process. I mean, it is really common and not the end of the world. I looked for any SAP Note related to the problem and I found the note 2734034 – SUM fails with Errors SQL11021, SQL14216 – cudf create script not found- SAP ASE for Business Suite. The solution worked so I documented it in my guide and continue with my life.

Last week I had to apply the SPS on the QAS environment. The process was exactly the same so I decided to apply the same workaround in the ASE database before applying the patches. This way I wouldn’t find the same problem during the SUM process so the downtime would be smaller. After applying the workaround I executed the command R3trans -d to check if the database was available and I found the following error message:

I thought ‘this is weird, it should work exactly the same as in the DEV system’. I never saw the message Caught signal 11 SIGSEGV (SEGV_ACCERR) Invalid permissions for mapped object while executing the R3trans. Let me explain you how I find the solution and which steps I followed.

How R3trans works

First thing we need is to know what is R3trans and how it works. R3trans is the R/3 transport program (surprise!) used to transport data between R/3 systems and for the migration between SAP releases. You probably knew this… Usually the R3trans command is not used directly and it is called from the tp command or other process. For example, when you start the SAP system the start process check if the database is available using the R3trans command:

If the R3trans output is an error the start process will fail and the SAP system won’t start. This is why I usually execute a R3trans -d before starting the SAP instances, I prefer to check by myself if the database is working fine in advance. R3trans uses the database client installed in the operating system. In case of Sybase ASE the client is installed in the global directory. In case of other type of databases it depends where it is installed.

Once the SAP system is started the work processes connects to the database using the database client. They will use the R3trans to connect to the database only on special occasions and usually through another process or command as tp command:

SAP Netweaver Architecture

The way R3trans connects to the database is the same as the work processes connect to the database and it usually depends on the type of database. It usually involve having encrypted files as SSFS, hdbuserstore key, etc. where we store the user and the password. The R3trans uses the DB host on the DEFAULT.PFL profile and it tries to connect using the encrypted user and password. Also depending on the database the DB host can be declared on the encrypted file, for example in DB2 and SAP HANA.

Caught signal 11 SIGSEGV

Once we know how the R3trans command works the next step is to investigate the error message. I tried to find any SAP Note with the error message Caught signal 11 SIGSEGV (SEGV_ACCERR) Invalid permissions for mapped object but it was impossible. On the SCN I found a couple of post but they were related to DB2 and in my case the database was Sybase ASE. There were some information related to this error message on *nix operating systems. It usually happens when one of the binaries being executed was corrupted or it had some kind of problem when the OS tries to access it.

Considering this I decided to follow this steps:

  • Upgrade the SAP Kernel to the same version as development (753 Patch 400).
  • Download the latest R3trans available.
  • Download the latest Kernel libraries related to Sybase ASE.
  • Check the database logs and try to find any issue over there.
  • Restore a backup done before the steps of the SAP Note 2734034.
  • Pray to God so I will have a vision about the error message Caught signal 11 SIGSEGV…

What to do when everything goes wrong

So I started with my test and nothing worked…

  • I upgraded the SAP Kernel to the same version as the development environment. I did this because I didn’t start with the SUM executing yet so the Kernel was not upgraded to the latest version. Since the development system was working perfectly I copy/pasted the same Kernel I was using in that system. Obviously it didn’t work…
  • I downloaded both the latest R3trans and Kernel database libraries available in the SAP Support Portal. When SAP released a new kernel version they pack all the Kernel binaries and libraries in the same file. When they solve an issue related with a part of the Kernel they release that part individually as a file. So if you download the Patch 400 then you can apply the Patch 417 related to the database libraries for example. it is quite common to update the R3trans or tp commands while upgrading a system or performing a DMO.
  • I restored the backup of the database that didn’t include the steps of the SAP Note 2734034. On Sybase ASE you can restore a point-in-time backup using sybrestore command. Unfortunately I still had the same issue…
  • I checked the database using the isql command and it worked perfectly. I couldn’t find anything related to the problem in the trace and log files. Basically every time I executed the R3trans command the database didn’t write anything related in the Sybase ASE Server log file. I also checked the operating system files and the system log. Nothing at all… Since I was a bit desperate at this point I decided to reinstall the latest patch in the database. The database was updated to Sybase ASE 16 SP03 PL05 HF1 a few months ago. I checked the update guide and the person who did it didn’t follow the SAP guide. He/she used the old procedure instead of using the saphostctrl command. With this command there is one option to force the saphostctrl to reinstall the database software so after a couple of minutes I tried again. Of course, nothing happened…

Pray to God. To the SAP God

At this moment I was completely frustrated. A problem with almost no information on Internet which I couldn’t fix. Did I find a unicorn? I was on my way to open a SAP Support Message as the last resource but then I thought again. The R3trans command uses the database client in order to connect the database. The database library is not included in the Kernel directory, it is actually on the global directory in the /sapmnt/SID filesystem. All the steps I followed didn’t do anything in the client directory so it was worth it to try a last option.

I downloaded the latest version of the Sybase ASE client from the SAP Support Portal. Since the database was running in Sybase ASE 16 SP03 PL05 HF1 I decided to download the client for Sybase ASE 16 SP03 PL06. Usually the database client could be on a higher patch than the database engine, I tried a lot of times in the past and it work. Anyway I checked the SAP Support Portal but I didn’t find any SAP Note related to incompatible versions of Sybase ASE client on different SP in Sybase ASE 16. I decompressed the client in its directory and I tried… It worked.

My face at that moment was like ‘are you kidding me? I spent 6 hours trying to solve this and the problem was on the database client’. I mean, I was happy everything worked correctly but I was frustrated because I couldn’t find the root issue. How the database client stops working with no reason at all? The steps I did on the database in the beginning didn’t change anything related to the database client. I still don’t understand what happened… Probably the database was updated but the database client was left on the old SP and when I changed the CUDF script in the database something stopped working.

Moral of the story

If I have to give you a couple of ideas about this experience:

  • Know your issue and how the different parts of the SAP system are related. In this case, the R3trans command uses the database client which uses the SSFS files to connect to the database. I didn’t thought about the database client in the beginning so it took my a while to figure it out.
  • If you are going to do something in your system have a way to reverse the operation. In my case I did a full backup of the database but it was a better idea to do a snapshot of the VM.
  • Have a battle plan and don’t do things stupidly. Try to tackle the issue step by step and document everything. In case everything goes wrong you will have the information and steps you did on a document you can upload to SAP.
5 (100%) 1 vote[s]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.