Using Mixxx as preparation software, exporting to USB

Dear Mixxx community,

I’ve recently upgraded from my old CDJ-1000s and bought myself a new Denon SC5000 Prime (2nd one to come when I can afford it), as I’ve finally become fed up of burning and carrying CDs. One reason I chose the SC5000 over, say, the Pioneer CDJ-2000 is because the SC5000 can perform most of the track analysis work on the player itself: this is important for me, as I run Linux and Rekordbox and Engine Prime are not well supported on Linux.

However, I am interesting in seeing whether Mixxx can replace Rekordbox or Engine Prime here insofar as being able to analyse tracks, and then export that to a USB stick in a format supported by professional players. This would allow me to use a more typical workflow, but still run as much free software as possible.

In my case, Engine Prime’s library (as written by my Denon SC5000) appears to be a series of SQLite database files. The Engine Library table schema looks fairly straightforward.

I’m not familiar with Rekordbox, but it looks like at least one person has already thought about elements of its format.

Is there any appetite for Mixxx to be able to export its library/crates/etc. to a USB stick?

Thanks in advance,
Adam

This feature could be developed, but it would be a lot of work to get it all working, particularly the waveforms. Mixxx stores the waveform analyses in separate files from the SQLite database. I have no idea how Engine Prime handles that. Even if you figure out where it’s stored in Engine Prime’s database, you’d have to figure out how to convert the information Mixxx analyzes to that format. BPM and musical key might be more straightforward.

Hi Be.,

I’m certainly prepared to analyse the Engine Prime format and document it. I’ll also be able to test out how well hand-crafting the Engine Prime format works, given that I have a SC5000 player to test it on. If I can share that, there will then be more people who can assess the complexity of converting Mixxx’s format to that.

If anyone else thinks this is worthwhile doing, let me know and it’ll spur me on :slight_smile:.

Writing to the Mixxx-devel email list would be a good idea. You could document the Engine Prime database format on the Mixxx wiki. Start a new page by going to the URL and editing the page, for example mixxx.org/wiki/doku.php/engine_prime_format

Someone else has reverse engineered the Engine Prime DB but used it to make proprietary software they are selling, so they probably wouldn’t be willing to help. :frowning:

The SC5000 Prime has a USB-B port on the back but I don’t think it is currently used for anything. I suspect Denon is working on implementing MIDI and/or HID control for a future firmware update. If so, it would be cool to map that to Mixxx.

The Engine Prime database format is not especially complicated, and I’ve documented most of the schema already. I’ll put this up online after tidying it up a little.

However, the really interesting information (such as hot cues, loops, beat grid, and waveform data) is all stored as binary BLOBs in the database. I’ve only just begun to look at them, but they look to be compressed upon initial glance and I’m trying to work out the format. I’ll keep working at it…

An initial draft of the Engine Library format is now up on the Mixxx wiki:
mixxx.org/wiki/doku.php/engine_library_format

This is an early version, so I’ll keep updating it. The main schema is documented, but the binary BLOBs in the PerformanceData table in p.db are proving elusive. They may even be subject to some form of DRM encryption, which will make the task significantly harder.

Well, it appears that the BLOBs in the PerformanceData table are not encrypted, but merely compressed (which makes sense, given the number of tracks one might want to be carrying out in one’s library).

I’ve managed to decode trackData and quickCues so far. Still working on the rest.

Wiki page updated:
mixxx.org/wiki/doku.php/engine_library_format

Haha, they are using Qt too! How convenient. Good work with this so far.

Managed to decode the ‘loops’ format too, after realising that some of the floating point values are stored with opposite endianness, ugh:
mixxx.org/wiki/doku.php/engine_library_format

Making good progress with beatData, and then only the waveforms to go!

Ok, a first attempt at beatData has now been uploaded onto the wiki page (got there eventually!)… In theory, even though waveforms are missing, this might be enough to start with some preliminary export functionality.

I may even consider writing a small library that Mixxx could link against to handle the serialisation etc.

Will keep at it…

Nice work mr_smidge. Sounds totally useful! Could be a real drawcard, too, especially if you can get it working with Rekordbox, since CDJs still industry standard, it’d be great to be able to rock up with a Mixxx-prepared USB stick and just go.

Thanks, naught101. Doing this kind of thing is hugely spare time-dependent, but never say never :wink:.

Latest update: I’m currently writing a C++ library to automate the reading/writing of the EP database, which perhaps Mixxx could make use of.

Update to a slightly old thread…

I’ve released an early version of a C++ library that can read/write the Engine Library format used by the SC5000, and put it on github: github.com/xsco/libengineprime

I’ve mailed the mixxx-devel group (as soon as a moderator approves my email :wink: ) to take the discussion forward with the devs, in case Mixxx can make use of it, or in case anyone wants to help with development of the library. Hope it’s useful!

Hello.

If you want to use a more direct communication with the developers, you should use the zulipchat : mixxx.zulipchat.com , and there, write to the #development channel.

So far, the mailing list is up, but is not seeing much movement recently.

Thanks, JosepMa - have now done just that.

I don’t have any Denon hardware currently, but I am saving up for the Denon DJ Prime 4 and have started placing my hotcues and loops for my library (at this stage about 240 tracks). I have been using Mixxx ever since I started to DJ and it sucks that I have to manually dial in beatgrid downbeats and hotcues into Engine Prime all over again, so this forum is getting me excited!

Unfortunately, I do not run a Linux machine and am a complete noob when it comes to anything related to GitHub or building applications from source code :stuck_out_tongue:

Hopefully this process will be made easier when the library code you made is brought out of its alpha stage. But until then, is there a decent tutorial video that explains how to build programs from source code easily on a Windows 10 machine without attempting to teach me everything about creating my own Git branch?

I feel your pain :angry:.

I appreciate that this isn’t an answer to your question, but in case it interests you, I’m working on integrating the above work into Mixxx over the next few weeks, so maybe soon the export-to-Prime functionality will come as standard.

I have built the libdjinterop library on its own in both Linux and Windows so far, but I haven’t yet got Mixxx integration working on Windows. So even if you could get a build working on Windows 10, there wouldn’t be any guarantee that it would work just yet. When I have a build working on my side, let me know if you’d be willing help test it!

Any updates on this? I’m very interested in this, as I’ve already started a huge project similar to this.

Mr_smidge or others with programming knowledge, send me a PM for further details.

I have the Denon Prime 4 and would be interested in testing this as well.

Hi djdemand,

This project is still moving, albeit slowly as I struggle to find enough spare time to work on it. However, your messages have spurred me on and I did some work today. I have forked Mixxx and created a branch, which is available here: github.com/xsco/mixxx/tree/lib_ … bdjinterop

The current status is:

  • Export to the Engine Library format works for tracks and crates.
  • Beat grids, cues, and waveforms are exported.
  • I’ve tested it on my SC5000 Primes, and even successfully played a couple of gigs using a library exported from Mixxx.
  • Made from two components: libdjinterop (the interoperability library) and Mixxx, which now uses libdjinterop.

Missing features:

  • Only tested on Linux so far. I can run a Windows VM, but I need someone with a Mac to help me out :slight_smile:.
  • Needs proper packaging before I feel happy submitting a PR to the Mixxx team.
  • Album art isn’t supported yet (I haven’t yet worked out the hash format in the EL DB).
  • Playlists aren’t yet supported (format is known, just need to implement it).
  • Some smaller missing features, e.g. average loudness on tracks not set, waveform summary.

I’ll reply to your PM as well!

Thanks,