ACSI2STM on the back of a Mega STE

This is my take on the definitive post on getting an ACSI2STM to work on an Atari Mega STE at the same time as the internal HD, so you can back it up on an SD card!

I have amended this post several times and will continue to do so as new solutions come to light. Last edit on 2024-10-27.

ℹ️ TL;DR

There is no one driver fits all solution but I have a workaround solution to recover the files from the internal HD. Scroll down to the Conclusion.

Using plain GemDrive

GemDrive is the default mode of operation of the ACSI2STM. It is some sort of “trick” that allows the ACSI2STM to expose the SD Card file system. It is handled by a native “driver” embedded in the ACSI2STM.

I believe this driver is loaded from “the boot sector” of the GEM drive, which is NOT provided by an SD card but provided directly by the firmware stored in the STM chip.

Attempt #0: try it in on a plain ST to make sure it works

On an STF / STE, you can just plug it in on the ACSI port, and boot up. You should see a driver load automatically (as explained above). The SD cards are then available as GemDrive “drives” on the desktop. If you insert two (standard FAT32 formatted) SD cards, they will show up as drives C: and D: (and a third one would E:) .

No drive/partition size limit. No need to even bother using ACSI drivers or anything like that. It just works. Awesome!

The lack of a proper DB19 connector on the unit is not so awesome however. But these connectors are no longer available and the way it plugs in works well enough, so fair enough, awesome enough…

DB25 connector compared to the fake pins on the ACSI2STM card

The image above shows how the ACSI2STM Compact card uses pin headers instead of a DB19 connector (like the blue DB25 connector shown on the left for reference of what it would look like if we had a DB19 connector).

This is what a boot screen with one card looks like:

ACSI2STM GemDrive boot message

The easy part stops here :/

❗ GEMDRIVE does not seem stable

Upon further testing with ACSI2STM on an Atari STF (TOS 1.04) and having a FLDR_500.PRG in AUTO, copying dozens of floppies onto the SD card in C:, the system ended up failing. This happened several times, even when copying only less than 10 disks since reboot.
But it seems correlated with moving/copying entire folders on the SD card (with many small files in those directories) in GemDrive mode.
I don’t think I ever had the same issue when using the ACSI2STM in strict ACSI mode.
I am inclined to say the GemDrive code crashes either in the ST driver loaded into the ST RAM, either on the STM firmware side running in the STM. Maybe more on the ST side since it seems to also affect no longer being able to read drive A:

TODO: there is a new firmware version. I need to re-test with the newer firmware!

Problem #1: the Mega STE Serial #2 is in the way

Above the ACSI port connector, there is the VME expansion bay and it has a Serial #2 port connector in it by default.

So, if you plug the ACSI2STM in here like you did on the STF, you run an intense risk of short-circuiting something between the RTC battery on the ACSI2STM board and the pins of the serial port, or even just touching the V+ of the battery on the ground of the connector… none of that sounds like a good idea.

So you need to extend the pins that came on the ACSI2STM.

Of course, the pin spacing is not a standard 2.54mm spacing:

Compare pitches between DB connector and a standard 2.54 mm header

So you need to break up your raisers into smaller bits that have enough tolerance to fit:

Raisers in max-widths of 4 pins at a time...

And check those pins and raisers. These are not the proper shape of the round pins that should go into the connector. No these have a squareICD section. Not so good for contacts…

ACS2STM now spaced away from Mega STE , thanks to the raisers...

I had to angle the card diagonally in order to get proper contacts. It wouldn’t work otherwise (ST would not see the ACSI2STM). My angling looks like this:

Note how the right side of the ACSI2STM is closer to the Mega STE case than the left side

This is very sensitive to touch. If it gets back to straight, it stops working. I need to rework the pins in some way at some point…

Well… a while later… I had enough of the flaky contacts and decided to take out the “Serial 2” port! It’s actually doable all from the back side (no need to open all of the Mega STE) (even if mine was actually already open):

Removing the Serial 2 cover plate and unplugging the ribbon cable

Here it’s possible to unplug the ribbon cable inside and then use the Mega STE with the ACSI2STM Compact and no Serial #2 port (which I don’t need right now):

ACSI2STM connected without raisers this time

Without the Serial 2 port in the way, the card can sit much closer to the Mega STE body and the contacts in the ACSI connector are much more stable:

And without raisers, we can be flush to the Mega STE case

Using GemDrive for SD and also accessing the Internal Drive

This brings more issues:

Problem #2: ACSI ID 0 conflict

At least when it has an internal HDD controller and an internal hard disk drive. In my opinion, having an internal HD was the whole point of having a MegaSTE.

That internal HDD controller is set to ACSI ID 0 by default. This is an instant conflict.

So you need to start by either changing the ACSI ID of the internal controller (which wasn’t properly documented anywhere) or you need to change the ACSI ID of the ACSI2STM card.

  • The former involves opening the Mega STE drive bay (one screw) and changing a couple of dip switches.
  • The latter involves soldering pin headers onto the ACSI2STM card and then place jumpers.

So it’s easier to start by changing the Mega STE internal ACSI ID. I set it to 1, 2 or 3 in the various tests below (remember it can’t go higher than 3 on this controller):

Mega STE internal HDD controller set to ACSI ID 3

Problem #3: The Atari AHDI driver aborts before ID #1 !

Atari’s ACSI driver v5.00, by default, queries each ACSI ID in order starting at ID 0. If it finds a drive it continues. If not, it stops. This is what a default boot looks like:

Well, now the internal drive is at ACSI ID #1 and the ACSI2STM card is at ID #0… but… wait for it… as long as you’re using it in the default GemDrive mode, all it does is create a conflict on ID #0, but once you free up ID #0, it doesn’t actually reply as a drive on ID #0.

So the AHDI driver thinks there is no drive and exits before recognizing the internal drive:

The solution here could be to load a third party driver, but after finding my old AHDI Driver disk, I found that it has a special folder called “MUNGED” that contains a variation of the SHDRIVER.SYS driver that polls all 7 ACSI IDs before giving up.

I could confirm this works by renaming SHDRIVER.SYS to SHDRIVER.PRG and running it. It found my internal drive at ID 1.

Then I had access to my hard drive and could replace the SHDRIVER.SYS with the MUNGED version of SHDRIVER.SYS .

At next boot, all went fine:

  • The ACSI2STM provides its GemDrive driver for thd SD cards.
  • The new SHDRIVER.SYS provides a hard disk driver for the internal drive.

For this particular boot, my internal ACSI adapter was set to ID 2, but you get the idea ;)

Problem #4: GemDrive and AHDI Partitions overlap!

What happens after this boot is that you have access to your hard drive partitions and to the ACSI2STM… BUT… the partitions overlap !

The ACSI2STM reserves C:, D: and E: for its own use (no matter if you have cards in each slot or not) but the AHDI hard disk driver doesn’t seem to know or care about it and also wants to use drive letters starting at C:. So:

  • you can’t access partitions C:, D: and E: on the hard drive. ❌ Those are the SD cards.
  • you can access your hard disk partitions starting with partition F: (in my case I have partitions all the way to J: … Don’t ask, I liked to containerize my workflows… ;)

Assuming you don’t want to run this continuously and you just want to back up your hard drive, how about back up F: to J: first, then reboot with just the hard drive, copy C: to E: on F: to J: (as space permits) and then finish the backup. Easy enough you’d think?

Problem #5 : Impossible to copy large files off the HD

With this setup (ACSI2STM GemDrive + AHDI Internal Drive) it’s impossible to copy a file larger than a certain size (which seems around 60KB, maybe 64KB) or else the whole system hangs with a busy bee forever (mouse still moves but can’t do much in busy bee mode).

Breakdown of tests at this stage:

  • Can’t copy >64KB from HD to ACSI2STM, ❌
  • Can’t copy >64KB from HD to A: ❌
  • CAN copy freely from ACSI2STM to A: or from ACSI2STM to ACSI2STM. ✅
  • Reboot without the ACSI2STM and everything works normally between Hard Disk and A:

So, it seems there is a read issue from the internal HDD when the ACSI2STM GemDrive driver is loaded, although that driver is not supposed to interact with the ACSI HDD. ❌

Could it be GemDrive screwing things up?

How about trying GemDrive + Other ACSI drivers?

See problem #11 in which I test GemDrive + ICD drive : it hangs :( ❌

Furthermore, it seems that when booting on GemDrive, just the mere fact of powering up the internal HDD will make GemDrive act weirdly.

Same thing: when booting with GemDrive + internal drive with NO driver, we can’t access GemDrive C: after boot. ❌

ℹ️ Conclusion #1

GemDrive may be useful when used standalone but it definitely doesn’t seem to be compatible with the presence of any ACSI drive (maybe even device) at the same time.

Using AHDI for both SD and HD

Problem #6: Getting the ACSI2STM to work in ACSI mode (instead of GemDrive)

To do that I used a different SD card and set the “Force ACSI” jumper on the ACSI2STM. Oh, by the way, this requires to solder extra pin headers onto the ACSI2STM board as it doesn’t come with them by default:

A jumper placed on the left pin header here will enable strict mode. The pin headers on the right can be used to change ACSI ID, but I'm actually not using them...

Booting with the internal HD, the new MUNGED AHDI driver sees the ACSI2STM at ID 0:

AHDI correctly recognizes the ACSI2STM in ACSI mode. ✅

But of course Unit 0 has no partitions yet, so the desktop only shows me the regular internal HD partitions after that.

I briefly tried to see if I could format or partition the SD card Unit 0 with the AHDI tools but it did not seem to be able to work with Unit 0…

Using ICD Driver for SD and HD

The ACSI2STM Github say they do their testing with ICD.

In it’s full name, it’s called ICD AdSCSI ST host adapter software, PRO version 6.5.5. You may find a donwload here.

Problem #6 continued…

So I got ICD Pro onto a floppy disk, disconnected the internal drive and booted with the ICD Pro driver tools floppy disk:

The ICD driver also recognized the ACSI2STM at ACSI ID 0 ✅ but again (as expected) no partitions (no drive letters appear on the desktop).

I then used the ICD Pro driver tools floppy disk to:

  • Partition the SC card (went without issues but it’s important to set “check passes to 0” to avoid insanely long bad sector check ). I created 2 partions (250 MB each) for testing.
  • Make the SD card bootable, still with the ICD driver tools floppy disk HDUTIL.PRG:

    This is the option to autoboot the driver from the ACSI drive

After that, I rebooted and it loaded the ICD drivers from the SD card partition C:.✅

So, I went on and rebooted with both the ACSI2STM (in ACSI mode) and the internal HD connected:

Good surprise: the ICD driver autoloads first (before the driver on the internal HDD) and detects the SD card partitions followed by the internal drive partitions, this time with NO overlap. The internal HD gets reassigned new drive letters. ✅

Surprisingly: the AHDI driver from the internal disk never loads. ❌ I’m not sure why:

  • either the ICD driver never lets the OS load the boot sector from the higher ACSI device IDs (ID 2 in this case)
  • or the AHDI driver self that is installed on the internal HD (ID 2 in this case) aborts because it sees there is already another driver loaded (the ICD drive)

❓ Question

Is it possible to have 2 different ACSI drivers loaded on TOS?
Maybe not…

Problem #7: Still slow to copy…

At that time, not only could I access all my GD partitions for backup but also: I could copy larger files…. but very very slowly. It seems the system still hangs when copying large files of the HD, but in a different manner from before… and it seems it manages to recover after a timeout of a few seconds.

But at least: copying of all files seems possible ! ✅

Problem #8: Could it be a 16 MHz or a Cache problem?

By default the Mega STE runs on “16 MHz with Cache”. This can be changed in the XCONTROL.ACC control panel.

With ICD drivers: changing to 16 Mz without Cache or even 8 MHz makes no difference.

Problem #9: the SD card formatted by ICD is not readable on Mac

ICD partitions, even below 32 MB in size, are not readable on Mac ❌ (and I guess not on PC either although I did not check)

Note that <32MB pure DOS partitions could also be created on a Windows machine (some recommend to do it on Windows).

Problem #10: Trying to boot with 2 SD Cards (GemDrive + ICD ACSI)

So, maybe I could copy the backups I have on my ACSI CD card over to my GemDrive card? Can I use both modes simultaneously (there are 3 card slots on the device…)

Boot sequence looks promising:

GemDrive driver loads...

Followed by:

ICD ACSI driver also loads...

But after reaching the desktop, none of the drives is accessible. It will either freeze or 💣💣💣… ❌

Problem #11: Change the driver of the internal HD

This could be done with ICD’s HDUTIL.PRG once again:

But it does not play nice with GemDrive:

  • Internal drive can now boot standalone:
  • The system can boot with ACSI2STM in ACSI mode + Internal drive.
  • But system can no longer boot when trying to use the ACSI2STM in GemDrive mode + internal HD :(
    It hangs on boot with “boot on C:” ❌

    System frozen at this point

Using HDDriver for SD and HDD

Problem #12: HDDriver Demo doesn’t recognize internal HDD

Here’s what I did to try out HDDriver:

  1. I downloaded the files from the demo version here and transferred them to a floppy disk.
  2. I rebooted with ACSI2STM forced to ACSI mode with a new SD Card.
  3. I manually loaded the driver by executing HDDRIVER.PRG
  4. I used HDDriver’s HDDRUTIL.APP to partition the SD Card.
    1. Select the device in the “Available Devices” window
    2. From the Medium menu, select Partition...
    3. Click the Compatibilitybutton
    4. Check TOS and Windows for “TOS & DOS” partitions, or check only Windows for pure DOS partitions (which should be limited to <32 MB). Leave the 2 Windows sub-checkboxes unchecked. Note that pure DOS partitions could also be created on a Windows machine (some recommend to do it on Windows).
    5. Enter MB sizes of your partitions and leave TYPE blank. TOS partitions should be <32MB. TOS&DOS can be up to 512MB (and still compatible with TOS 1.04). The demo version will limit the partition sizes it creates to 2.1 MB

      Yep, I swapped displays in between... This whole testing lasted over a week...

  5. Then I had to reboot and execute HDDRIVER.PRG again:
  6. At that point I could access my partitions on the desktop:
  7. Still with HDDRUTIL.APP, I installed the driver on that device (i-e: SD Card):
  8. I rebooted and the driver loaded.
  9. Then I went back into HDDRUTIL.APP to change the settings:

    Contrary to my screenshot, you MUST NOT go into the “Tools” Menu. Instead do this: In the SettingsMenu click the Devices and Partitions... command. Then turn on devices 0.0, 0.1, 0.2, 0.3 for ACSI IDs 0 to 3 on the ACSI bus #0 (which is the only one you have in the Mega STE).
    Optionally:
    • Settings-> Removable Media... : Set Maximum Sector Size to 8192 (if you use that in large partitions)
    • You can drag and drop the 0.x buttons if you want to change the boot order !
    • You can double-click the 0.x buttons if you want to define LUNs… (but by default LUN 0 will be active and that is probably all you need).
      However…
  10. After reset, the driver correctly scans all 4 ACSI IDs:
  11. After power down, connect internal driver and power up, the internal drive was recognized:

So far, all seems good… but it actually isn’t:

Problem #13: Garbled content! DANGER ZONE !

I copied a few files from the HD to the SD Card. The demo again limits writing to drive C:. Other partitions will be read-only. And as C: is only 2 MB max, the test is limited… but copying a large file seemed to work without issue (except the demo driver is artificially slowed down, so it’s a bit unclear if it’s slow because of that or if it’s slow because of something else… but yeah… it seemed to be a bit faster than with ICD… )

Then I moved the card over to my Mac, and…

  • At first glance it seems to works, it can see the partitions
  • Then I can see the files I copied on C:
  • And when I open a text file, the beginning is correct but the end is garbled !!! ⚠️🚨 DANGER !! You could lose your work! ❌❌

Problem #14: Mac/PC compatibility with HDDriver

When I tried to go back to the Mega STE and boot with the card, it bombed badly 💣💣💣💣💣💣💣:

So I think this method cannot be used for transferring files back and forth… ❌

I made a second try, just by reading from the card, not trying to write again. Again, the card became unusable.

Note: It may be because the Mac automatically creates unexpected hidden files (and with long filenames too) but HDDriver is a recent and maintained driver. I expected a bit more robustness…

Conclusion on HDDRIVER

❗ Conclusion on HDDRIVER

While HDDriver probably has the most comprehensive configuration interface of the bunch, it seems downright DANGEROUS to use as it seems to corrupt files in copy operations. ❌
It also doesn’t support Back & Forth with MacOS. ❌

I also contacted the author Uwe Seimet. His site lists ACSI2STM as “not officially supported but some users reported success”… but he didn’t care much about the photos of documented data corruption that I sent him and replied “there is no reason to believe other users did not have success using it with their ACSI2STM”… yeah… let’s be positive… let’s look only at success cases… if we have positives then it negates the negatives, right? right?

Moving on…

Peter Putnik’s drivers

Problem #15: Trying to make sense of all the files…

I bought Peter Putnik’s drivers for 30 € and he sent me 10 different ZIP files:

I thought I bought a driver. Instead I got a puzzle...

Good luck making sense of that… If you enjoyed the organisation of Putnik’s website, you will love the way he packaged his drivers and tools.

Each ZIP file contains various additional files with short 8 character names… After unzipping, there are variously named TXT files that more or less try to describe what the file is about, but they all sound the same. Some files are the exact same base with a few changes here. The text files don’t clearly document what each driver does and does not support. Additionally they are written in broken English and without proper paragraphs. Newlines are randomly distributed. Excellent user experience if you like puzzle games. ❌

I never got a proper guide to what to use for what but after several days of repeated inquiries, Peter Putnik graciously told me I should use ACSIDRV but I never got a real answer as to why this one and what all the others are for…

Anyways, here’s what I did:

  1. Use PPP12A.PRG to partition an SD card:
    • Select the ACSI 0 button . (Master/Slave/Tw IDE are for IDE drives.) If needed change ACSI \# and reclick ACSI 1 for example to load vendor name and current partitions (works only for DOS compatible partitions).
    • Enter partition sizes, check Free space.
    • The default type is C16which is a TOS&DOS type (FAT16A). (TODO: Maybe try pure DOS partitions later?)
    • Then PARTIT. and INIT all button
    • After partition, click ACSI 0 again to see the actual created partitions
  2. Use AAC_102V.TOS to install auto boot driver
    • Note (according to the readme files): contrary to other HD drivers, PPDRIVER does not install the boot code in a specific partition (I guess like SHDRIVER.SYS) but in the free space (30KB) just after the MBR (reserved sectors). This allows to select at boot time which partition you want to use as C: from the list of all the available partitions on the media." ✅
    • “Option for uninstalling driver does not exist, because it makes no sense. Driver will be simply deactivated if you install another driver on the media.”
    • “Note that PPDRIVER does not have any configurable option . It does not support XHDI and does not include FOLDER100. This was done on purpose not to waste any space for gamers .”
    • It’s unclear whether the driver can only boot from TOS&DOS partitions or also from pure DOS partions.
  3. Reboot: it works.
  4. Turn off, connect internal HD, reboot: it works as expected:

So once again, it looks promising… bit it isn’t…

Problem #16: This one too fails on large files…

Trying to copy 17 files from HD to SD:

It will hang for a while (like 30 seconds) when it encounter this (or any) larger file (the limit seems to be at 64KB):

Then it will say it cannot create the file on the destination disk:

The end result is pretty weird:

  • The large file is missing ❌
  • The first file is ALSO missing (there was no error message about this one) ❌

Problem #17: Another PUTNIK driver? It gets worse…

I tried a more recent driver V1.03idi which says it’s from 2022:

I followed the same procedure as above and the results were almost the same, except that instead if saying it can’t create some files, it said nothing and still did not copy them, which is probably even worse. ❌

Then after a while, I found that the TEST folder that I created on the C: partition of the SD card changed into something else:

This weird file is actually a folder...

So here again, we have silent data corruption! ❌

Conclusions on PPUTNIK’s drivers:

ℹ️ Conclusion on PPUTNIK’s drivers:

  • Can boot on SD ✅
  • Can read the card on Mac (tested on partitions as large as 250 MB) ✅
  • Can boot again after read on Mac (The Atari ST will display the junk files created by Mac OS) ✅
  • I can’t fully trust it with file integrity either but it may depend on which one of the driver bag I’m using… ❌

Problem #18: Trying to get MacOS to not touch the files…

I found this on the internet:

defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool TRUE

I applied it but it doesn’t seem to help. Mac OS continues to pollute the SD card. ❌

Preliminary conclusions

ℹ️ Conclusions backing up internal HDD to ACSI2STM:

  • GemDrive may be useful standalone (but v5.0a did crash even standalone see Attempt #0). ❌
  • GemDrive used in conjunction with an ACSI HDD seems like a dead end. ❌
  • All ACSI drivers seem to have issues with large files when ACSI2STM is present on the ACSI bus (even though, of course, it uses a different ACSI ID)
    • The original Atari AHDI driver cannot copy large files from HD to SD. ❌
    • The ICD driver can copy large files from HD to SD ✅, but the resulting SD card cannot be read on Mac (doesn’t use TOS&DOS partitions). ❌
    • The HDDriver seems to be able to copy large files but it corrupted them in the process (DANGER!) ❌
    • Peter Putnik’s Driver (at least the one he told me to use) cannot copy large files from HD to SD. ❌
  • Considering that 2 drivers have issues with the ACSI2STM, it raises doubt on the quality/implementation of the SCSI protocol in the ACSI2STM. However, the drivers do not return proper error conditions which would lead GEMDOS to display an error message, and that definitely puts doubt on the quality of those ACSI driver’s code, especially when it comes to handling error conditions. ❌

Anyways, considering all this, the hack solution I found is:

💡 Hack solution for backing up internal HDD to ACSI2STM:

  1. Make an SD card using ICD and copy all files from the HDD to this SD. ✅
  2. Disconnect HD and remove the “ICD” SD card.
  3. Make another SD card with P Putnik’s Drivers.
  4. Reboot with the ACSI2STM having the “PUTNIK” SD card in slot #0 and the “ICD” card in slot #1. Then copy all files from “ICD” to “PUTNIK”.
  5. Bring the “PUTNIK” ICD card to your Mac/PC and you can copy in both directions. ✅

Also, once backup is done, the problem simplifies to just using the ACSI2STM without crashes:

ℹ️ Conclusions if you’re not trying to back up:

  • You can use GemDrive until it crashes. Your mileage may vary. At least it does not silently corrupt your files.
  • In ACSI mode:
    • Atari AHDI tools cannot format the SD card. ❌ Did not test if it can be used as a driver once the drive is formatted.
    • ICD driver can be used to partition and to access the card. ✅ ICD partitions can not be used for data exchange with Mac. ❌
    • HDDriver can be used to partition and to access the card. ✅ The driver crashes if the card has hidden files auto-generated by a Mac ❌
    • Putnik drivers: hard to use but can work to partition the card, to access the card and to exchange data with Mac ✅

TODO

  • Try ACSI2STM in GemDrive mode again with latest firmware v5.0c
  • Try ACSI2STM in ACSI mode again with latest firmware v5.0c
  • Try EmuTOS with its built in ACSI driver + ACSI2STM v5.0c