Soundjack article

Synopsis

Impacts, of the COVID-19 pandemic, on in-person gathering of musicians, has accelerated development of online musical collaboration software applications. The suitability of such software for ensemble playing and remote lessons is dependent upon parameters of audio fidelity,  low-latency, numbers of participants supported, ease-of-use and cost.

This paper provides a quick-look at the experience, to date, of a small team,  using one of these collaboration applications – SoundJack.

Background

The COVID-19 pandemic has had devastating consequences for live music activities, amateur and professional alike. In-person ensemble practice, performance and education have all been seriously impacted. Although, at this writing, light may finally be discerned at the end of this long tunnel, it will still be some time before a return to normalcy arrives, and the traditional forms of musical collaboration we cherish may resume.

Attempts at virtual music collaboration, using collaboration software applications, have met with limitted success, due to poor performance with respect to audio latency. Successful ensemble playing requires that all players hear their partners playing in close, and consistent, synchronization with their own instrument or voice. Telecommunication networks, designed principally for non-realtime data traffic (e.g., document transmission such as email and Web pages) challenge interactive audio and video communication.

Avaliable collaboration apps suffer from a variety of difficiencies such as delays, dropouts, stalling and/or mediocre fidelity. While often tolerable for speech communication, these faults are unacceptable for simultaneous music collaboration. For example, online lessons require that either the student, or the teacher, mute when playing together, less one of them hear the other play… umm.. shall we say off-the-beat?

To address these shortcomings, speciallized music collaboration applications have appeared. Their creators employ a variety of techniques, that purport to overcome the worst defects of playing together through our computers. But did they succeed? And, if so, is that success transferable to musicians, who may lack deep computer technical skills? What is required?  Is it affordable? A team of Glen Legere, Gregory Valenzuela and John Thier set out to evaluate one of these tools and attempt to find out.

Three caveats: First, this project is a work-in-progress, and this paper is intended  to report only what we know so far. Second, it is not a comparative analysis. We’ve only worked with one tool – SoundJack – and make no assumptions or assertions, positive or negative, regarding its competitors. Finally,  while aspects of configuration and operation of this tool are discussed here,  this is not a complete”How-To” guide to SoundJack. Detailed documentation already exists, courtesy of Dr. Ian Howell at the New England Conservatory of Music. SoundJack: Comprehensive User Guide (referred to herein as the ‘NEC Guide’ ) provides considerable information regarding SoundJack installation and use for some specific hardware/software computing configurations. Click through the above link to read the guide and see other documentation and videos to help you get started.

Furthermore, an expanded version of this article, soon to be available,  will provide details regarding the configurations our team utilized, as well as considerable background information, which, hopefully, will help to demystify the process of getting SoundJack up-and-running.

So, what, exactly, is Soundjack?

SoundJack is a global software platform for live music collaboration. It allows multiple participants to “hook-up” online, and play in ensemble groups, with reasonable fidelity and low-latency. The size of such groups supported, is dependent on the computational and network capabiities  available at the user’s home, office  or other playing environment. SoundJack is free.

The SoundJack software has two major components which interact to establish, configure and operate online playing ensembles: a centrallized web-based component, (hosted in the EU), and a real-time application, which runs on each user’s computer. We’ll refer to these  as, respectively, the SoundJack Website, and the SoundJack App.

The Website, accessed through a user’s web browser, lets users find their partners and adjust settings of their real-time apps. These settings optimize the transmission quality to and from each of their peers. Don’t be alarmed by this – it’s easy. In addition. a global chat room is provided. The Website is also the place from where the SoundJack App is downloaded  to the user’s computer. 

The real-time SoundJack App is the piece that actually sends music between the playing partners. What is meant by “real-time”?  Well, there are a number of technical definitions, but in the words of a mentor to two of the authors: “Real-Time is Fast Enough”.  Collaboration software on the internet is usually not “fast enough”, for a variety of reasons. Our expanded document will discuss some of these in more detail, but, essentially,  two stand out:

  1.  All data traffic is routed thru central server platforms, rather than directly between users.
  2.  Software network protocols (the handshake agreements that computers use to make sense of what they are sending and receiving), are biased toward reliability, rather than low-latency.

SoundJack gets around the first obstacle by utilizing peer-to-peer (PTP) technology, where each user has direct connections to his/her partners, similar to the approach taken by  music file-sharing applications. Ethics aside, they work well, allowing fast transfers of lots of data. 

The second issue is addressed by using a fast data transfer protocol , which does the opposite of what other apps commonly use – it’s bias is toward speed, rather than reliability. That may sound a bit frightening, but it’s not – we’ll have more to say about that in our expanded guide. For now, just be aware that reliability, in the network sense, is not the same as security. SoundJack’s protocol selection does not pose a network threat to your computer.

Once installed, you don’t have to do anything with the real-time SoundJack App except start it up. User interactions to the App are made with your browser through the SoundJack Website.

OK, So What Do I Need to Use SoundJack? (aka, the dreaded “Minimum System Requirements”

The NEC Guide includes a chart, which provides a sliding scale of usability for the various hardware and software elements needed for SoundJack. We won’t repeat that here, but speaking at a high level, this is what you need: 

  1.  A host computer and operating system. SoundJack supports Windows , MacOS and some Linux varients.
  2.  A microphone of reasonable quality. Microphones are classified as High Impedance (Hi-Z) or Low Impedance (Lo-Z).  Lo-Z microphones typically come with “balanced” outputs, and are distinguished by their 3-pin, XLR connectors. Practically all microphones now made for recording  are balanced,  Lo-Z, and, if you’re buying a new mic, this is what you want. Hi-Z mics can be used with some restrictions; we’ll go into that in more detail in out companion guide.
  3. An external, bidirectional audio interface,  unless the host computer includes audio support of sufficient quality (Macs may suffice, PC desktops, laptops, etc. may not). By audio interface, we mean units which convert analog audio signals, like microphones , guitar pickups, and line-level (think CD player) outputs, to digital signals, and vice-versa.  Suitable audio interfaces include PC sound cards, which are added to desktop computers (typically on the Pci bus). But a better option for most people today, are audio interface units which connect to one of your computer’s Univeral Serial Bus (USB) ports. These are relatively inexpensive , and provide a wealth of capabilities beyond what is needed for SoundJack. For example, a unit made by Behringer (UMC202HD) provides a USB interface, two channels of microphone preamplification, MIDI I/O, channel mixing and even comes with Digital Audio Workstation software. A complete small home recording studio. $79 from music retailers like Sweetwater. 
  4.  Internet Router. The box you either plug your computer into, using an ethernet cable, or connect to with WiFi, to access the Internet, or other local computers. However, you’ll  need a minor change to the router configuration for SoundJack to operate (more on that in a bit – don’t panic).
  5. Broadband Internet Service (assuming your playing partners are remote, i.e., not just in other locations on your local network, as on a campus network). Some of us are fortunate enough to have true broadband service, say, 25 Megabit/sec and up. Less-capable offerings, such as Verizon’s Digital Subscriber Link (DSL), clocks in at about 6 MB/sec. And some still suffer with ancient dial-up internet access. The NEC Guide lumps DSL with dial-up in the “not-really-usable” end of the scale. That said, one team member (John)  operates successfully on DSL , but, as we discuss below, that may be a limitting factor to our results. 

Getting Started – Network Configuration

Before setting up SoundJack on your computer, two changes need to be made to your network configuration. Our companion document wil provide an in-depth explanation, for anyone interested, but here are the essentials:

  1.  A network configuration setting, must be made which guarantees that the same  Internet Protocol (IP) address be assigned to your computer, every time it starts up. This is known as a Static IP Address.
  2. A router configuration setting must be made, to forward SoundJack’s data packets between your SoundJack host computer, as specified by its Static IP address, at port 50050, and other Soundjack computers. This is known as port forwarding. (Our companion guide will provide a detailed explaniion of IP Addresses and ports).

The above steps must be performed for each SoundJack host you stand up, even if they operate through the same router.

It’s possible that your Internet Service Provider (e.g., Verizon, COMCAST, etc.) can do this for you, either remotely or with a service call – you can contact them and ask. But if they can’t or won’t, you have to log into your computer and/or router and do it yourself. 

These configuration changes are easy – if you know how. Aye, there’s the rub – each router and computer operating system has it’s own way of doing these things. They’re not very different, but, “different isn’t the same”  – which means that providing instructions that cover every case isn’t practical. What can be done is to provide speciifics for some common computer/network environments that will  give users a good sense of what they need to accomplish. They can then work with their ISP to determine  the procedure for their particular situation, if it varies from the examples. Some examples are given in the NEC Guide. Our companion document will provide detailed instructions for configuration of the three equipment sets used by the team. In addition, user manuals for routers invariably give port forwarding instructions. These are often available online, or from your ISP. Finally, web searches like  “Setting Static IP in Windows 10” or “Setting Port Forwarding on D-Link DSL 2750B router” can often yield all the information required. 

 

Getting Started – SoundJack Setup

Once you’ve completed your network configuration, you’re on easy street. Now you can set up SoundJack itself.

  1.  With a web browser, register at https://www.soundJack.eu. Registration is pretty much like registering at any online vendor. Not much information is asked. You will be asked to create both a user ID and password. We should mention here that for SoundJack operation (not registration and App download), Safari cannot be used – it’s incompatible with SoundJack.
  2. After registering, log into the SoundJack website (same web address). At the Download tab, you’ll find options for Mac, Windows and Linux. Download the appropriate SoundJack installer and follow the typical installation procedures for your computer type.
  3. If all goes well, you should now be able to start the real-time SoundJack App on your machine. For Windows, opening or double-clicking soundjack.exe will  start a command shell that you can look at to monitor its operation. For Macs, you get an icon to push, but not much else (there’s a pull-down menu to stop it). In all cases, the real user interaction will be through the Web application.  

Using SoundJack – The Stage

The primary control panel for SoundJack is the Stage. From the login, clicking the Stage tab brings up a multi-panel  web page. 

The left-hand panel is a column of configuration controls. More on that in a bit.

The Center panel  is a list of users currently on SoundJack. By default, this is a global list – everybody in the world currently using SoundJack. It’s easy to set up  friends or groups, and limit the user list to those. The important thing aboout this panel it that it’s the place where you connect to fellow players, receive feedback on the quality of your connection, and set an important configuration control, specific  to that connection.

The right panel is a chat room. Feel free to participate! Or not.

Using SoundJack – Setting up an Ensemble

Details of establishing connections to your playing partners aren’t difficult, but are beyond the scope of this paper. For that, refer to the NEC Guide and, when it’s published,  our companion document. But briefly, here’s what you do to set up a playing session with one or more partners:

  1.  Start the SoundJack real-time application.
  2.  Using your browser, log into the Soundjack Web application, and navigate to the Stage.
  3.  Using the configuration panel on ‘Stage Left’, set initial configuration parameters. These do things like let  you select which audio devices, operating with your computer, you wish to use as inputs (i.e., microphones) and outputs (speakers and headphones). You can also mix in your own instrument (or voice)  as feedback along with sound from your partners (e.g., for recording). Most importantly, here is where you set all but one of  the quality-of-service (QoS) parameters – configuration controls which tailor sound latency and fidelity.  In practice, after initial setup, you will only need to adjust three of these (two on this panel and one other) when  establishing  a playing session. The NEC Guide provides recommended settings for all configuration controis, and our companion document will give you the settings we found useful.
  4. Find a playing partner on the center panel. Double clicking the icon for your partner will send a connection request .  That will  launch a pop-up window in your partner’s browser which allows  your request to be accepted (hopefully!) – or not. 
  5. If all is reasonably OK, both players should now hear each other. The connection may not be perfect, however. If the sound either way is garbled, adjustment  of three QoS parameters (two on the configuration panel and one near your partner’s connection icon) will be necessary to smooth things out. These adjustments are all menu selections – you don’t have to enter any magic numbers  The NEC Guide provides some strategy for these adjustments, but our team found trial-and-error to be the best course.
  6. To add partners to your ensemble, just rinse-and-repeat steps (4 )and ( 5).  The big caveat here is that the number of players you can add will depend on your respective computing and networking capacities.

 

Team Experience

Our evaluation team ncluded  three individuals, each with different computer, audio interface, microphone and network/router enviroments.  Glen is located in eastern MA, Greg in central NJ and John in western MA. 

Network configuration changes (Static IP Address and port forwarding) went quickly and smoothly for all team members. Details will be provided  for all three setups in our expanded guide. 

An initial connection attempt between Greg and John was unsuccessful, with severe garbling of the audio (at best this seemed to resemble a ‘motorboating’ (‘put-put’) sound, and at worst complete loss-of-intelligibility. A second attempt, on the following day, resulted in complete clearing of the problem, by simple adjustment of one of Greg”s QoS parameters. Latency was imperceptible. Audio fidelity was fine.  Attempts to optimize the connection further, with trial-and-error adjustment of the three QoS parameters, showed no improvement, nor was any needed.

A three-way session with all team members showed similar results for each duo (Glen/Greg, Glen/John, Greg/John). In each case, high-fidelity audio with no obvious delay was achieved, with, at most,  only a few minutes spent ‘tweaking’ the QoS settings.

Unfortunately, this success did not extend to trio sessions. Once all three connections were established, it was found that while two of the three players could hear one other well, the third was severely garbled. Adjustment of the SoundJack parameters did not resolve the problem.  While no definitive cause has yet been established, some suspicion lies with the limited bandwidth of John’s DSL service. It is also possible that the trouble lies with the performance of Greg’s older Windows computer. The team will continue to investigate. 

That said, it should be noted that, since the initial experiments, Glen and john have conducted successful duo practice sessions, with excellent audio fidelity and no latency concerns.

Summary

The Pandemic has clearly revealed a need of technology for effective  real-time, online music collaboration. The computing software community has attempted to meet that need. We can say with confidence, that, within limitations , not yet fully defined, the developers of SoundJack have succeeded.  At a minimum, live duo performance, across distance, heterogeneous  technology platforms and differing network providers, works and works well. We anticipate that with additional research, we may identify, and overcome, the performance issues encountered and expand the size of our online ensembles. 

To that end, we encourage others to join us!  The more participants we have in our ongoing experiiment, the greater the opportunities to see what this exciting new tool can really do. Glen and John are beginner/intermediate gambists and would appreciate playing  online with others.

We all hope that the constraints COVID has placed upon our in-person gatherings, to teach, practice and perform, will soon be behind us. But that need not be the end of online  playing communities. The concept of a “distanced partner”, joining with an in-person group is now viable. Those prevented from participation in consorts due to travel  time, inclement weather or health issues, now have an opportunity to play with their fellow musicians.  SoundJack, and apps like it, would now seem to be indispensible  tools for musicians in today’s world.

For technical assistance, feel free to contact:

Glen Legere galegere@comcast.net

John Thier kc1zt@yahoo.com

Greg Valenzuela gvalenz@verizon.net