My First VJ outing with VDMX

My First VJ outing with VDMX

So, on saturday I had my first public performance with VDMX. I was going from my Macbook Pro to an HD projector through a byzantine array of converters since there is no publicly available mini-display port to HDMI or component adapter. My signal was squashed to SD over RCA, but that was ok, since none of my source material was HD. Monoprice.com is supposed to have a Mini-display to HDMI adapter ready by the 15th of this month, so we'll see how that works out next time.

Overall I was pleased with VDMX. My setup had to run itself, because I was going to be busy playing keys with my band, and running around like a headless chicken for the rest of the show. I made three layers, one on top using an OpenGL add mode, two on bottom, transitioning back and forth using a quartz composition 'xerox' wipe. All the layers had clips cycling through them using a step sequencer. Since I had my computer in a balcony VJ booth, I was able to include a video input as a clip in my bin, and then angle the camera in my MacBook screen down at the audience so occasionally there would be a crowd shot projected, which seemed to give the audience a kick.

My visuals did get a little washed out due to so much simultaneous stuff going on at once with that OpenGL add mode, but that's something that shouldn't be too difficult to improve on next round. I may also try to add touchOSC controls from my iphone next time, although I am pretty content to just let the thing run itself so I don't have to worry about it. If I was vj'ing for another band I would probably add a midi controller and work that way.

I'll post my vdmx project file in a bit, as well as some video of the show when I track some down.

Digital Vs. Analog Mixing. Which is Better?

Digital Vs. Analog Mixing. Which is Better?

Written by Ean Golden on December 7th, 2009 77 Comments

There has been a great debate taking place in forums around the digital DJ web. What is the best way to mix songs while DJing: externally on an analog mixer or inside your DJ software? Many DJs — myself included — have always assumed that mixing on an analog hardware mixer is the best option if you have the choice. To put that subjective assumption to the test, DJ TechTools interviewed numerous experts in the industry, including audio engineers, software developers and mixer manufacturers to get down to the bottom of this important question.

THE BIG QUESTION

Let’s assume you have DJ software, such as Traktor, that lets you mix internally or externally. When mixing internally, all of the audio signals from the decks are summed inside of the software and then sent out to the sound card as a single master output. All level controls, EQ and any distortion happens in the digital world before being converted to analog. When mixing externally, each deck’s audio is routed to separate outputs of your sound card. These outputs then connect to a DJ mixer, where the summing happens on the mixer in the analog world.
Before we dive into the audio quality question, each of these methods offers a few clear functional benefits.

At this stage, the contest is fairly evenly weighted with each option having clear pros and cons. The argument gets skewed when subjective experiences of audio quality start to get thrown in the mix. Statements like “external mixing sounds better” trump any pros/cons list, resulting in people making decisions based on assumptions rather than facts. So, to help us all understand the difference once and for all, we got in touch with the people who know this stuff inside and out.

THE ANALOG SIDE OF THE STORY

Our first stop in the analogue world is Rane, a Seattle based company known for making highly respected DJ mixers for more than 20 years. Rane makes analog mixers that sound good no matter how hard you push them, and digital mixers as well. That raises the question of how they get a TTM-57SL, which mixes all of the signals digitally on a DSP (just like software) to sound as good as their analog TTM-56. Steve Macatee, director of new product development, who has a lifetime of experience in designing quality DJ mixers, explains,

“Most people are used to the sound of CDs, which are 16 bits and 44Khz sample rate, but what we do in the TTM is far better than that. It’s a 24-bit world and 48khz sample rate, which increases the dynamic range and lowers the noise floor.”

A lower noise floor and wider dynamic range means that it’s much harder to distort a signal, providing the DJ with more head room. What happens when you do distort the audio though? Everyone is going to push the signal into the red at some point in the night, so isn’t that when analog starts to get more forgiving than digital?
Steve again corrected my assumptions,

“That’s not necessarily true. Because digital mixing technology has evolved over the past 20 years, you have digital converters that clip in very friendly ways. It really depends on the op amps and the circuitry you’re using. You can design an analog product where the audio clips the voltage rails and sounds like hell. [Editor's note: think Pioneer DJM-500.] You can also design an analog summing buss that doesn’t clip and sounds pretty nice. Whether analog or digital, you can do a good job or a bad job of it. What really matters is the quality design of the summing circuits and the dynamic range you get on the output.”

Okay, so the quality of circuits really depends on the designer, but don’t analog mixers offer certain soft limiting features that keep a DJ mix sounding good even when they are pushed into the red? Not quite:

“That’s a fun question, because in the analog world, a lot of our DJ products have outputs stages with little fixed limiters that are trying to keep people happy and not make the signal sound horribly distorted when the DJ turns the sound full on. When you do the same thing in the digital world, you have a plethora of possibilities, because you can do things in a much fancier way without taking up crucial board space (inside an analog mixer) because now what you have is a software problem. The only limit is the amount of memory your DSP has, which limits how fancy of a limiter you can create and how the audio circuits are built.

So in reality, if you’re comparing apples to apples, then digital mixing has the potential to be more flexible and forgiving than analog… if it’s designed properly. Since each product is going to have different characteristics and qualities depending on how well they are made, you really need to use your ears and put both software and analog mixers to the test before judging.

DIGITAL MIXING

Traktor’s Take
We got Native Instrument’s take straight from none other than Stephan Schmitt, who created professional analog mixing consoles before founding NI.

“First of all, it is important to understand that audio summing circuits in a mixer or a similar hardware device are designed to combine electrical signals in the way of a very straightforward mathematical addition. In that sense, the internal mixing in Traktor – or any other properly designed audio software – is actually superior to a hardware mixer because it combines the audio signals in a mathematically precise way, and without introducing any noise.
One important aspect of digital summing is internal headroom, and this is where the audio quality can potentially suffer. Traktor overcomes this by using 32-bit floating point calculations, so headroom is practically unlimited (with 750 dB), and the mixer in the software can never produce unwanted clipping artifacts. This is not to be confused with the master output stage of the software that drives the audio interface — you need to take care not too overload this stage, and that’s why Traktor also offers an optional master limiter.
In a general you could call software mixing — or basic digital summing — “perfect” in terms of sound quality, and there is not even any DSP magic in the form of especially sophisticated algorithms required.
Obviously, certain analog mixers can color the sound in a way that some people find pleasant (though most mixers are much more likely to degrade the sound unless they have a really high-quality signal path). To cater for this aspect, Traktor offers the choice between a generic, “mathematically ideal” EQ on the one hand, and a selection of EQs emulated from actual high-quality hardware mixers on the other hand. These analog-modeling based EQs even color the sound in a subtle way when all bands are set at the zero position, in the same way as the analog counterparts.
When it comes to audio quality of DVS systems, the signal quality of the audio interface is a huge factor: the quality of the converters, signal-to-noise ratio and sheer output level. Also, software DSP functions such as digital effect algorithms and time-stretching do have a huge influence on sound quality, and require a lot of engineering effort and expertise. The difference in sophistication between a “standard” chorus or reverb algorithm and one that actually sounds good is enormous. Audio summing however is a very straightforward and comparably trivial part.

The Itch Side Story

What about the Serato camp and its relatively new Itch product? How are they doing summing in the digital domain?

“The Itch mixer is designed to have enough headroom to effectively mix songs together without hitting full scale at the output.  Itch also has a master limiter on the main mix bus that will kick in if a DJ gains everything on full or is using high gain and lots of EQ, this limiter is to avoid clipping before being sent out to the sound card.  We have a feature on the setup screen called overdrive where you can adjust how much headroom is available in the mixer before the output limiter kicks in. By default it’s set to it’s safest setting, which offers the cleanest sound. Some people like more limiting and a crunchier sound, so want this turned up.  We also offer 6 or 12 DB boost on the EQs as an option, so a DJ can choose how they want it to sound.

STRAIGHT TALK

Equipment and software developers have a special expertise to be tapped, but their proclamations can also be biased toward the advantages of the particular gear they make. For some balance on this eternal debate, we turned to experts in the music technology education and journalism fields.
“Experience has indicated that analog summing is superior in creating separation and a better stereo image,” says Paul W. Hughes, a national manager at the SAE Institute chain of multimedia schools. “Tracks that have ‘a lot going on’ tend to lose definition when summing digitally in the box,” he continues. “Fundamentally, your [software] uses math and algorithms to blend the signals. That’s not how your ears process sound, so the more it has to sum, the more the sound will ‘feel’ as though it’s tanking out, losing punch, drive and energy. Pushing it louder will only result in distortion.”
George Petersen, the long-time editorial director of Mix Magazine and an active producer/engineer, says “Really, analog summing is the best way to go, but in most situations, the difference is pretty subtle.” Petersen doesn’t discourage anyone from mixing in the box, but he emphasizes the importance of a high-quality audio interface if you’re going to get the best sound using digital summing.
The quality of the digital-to-analog (D/A) conversion of your interface should be excellent, but so should the analog circuitry that amplifies the the sound after the D/A conversion. “Not only does the converter chip have it’s own sound,” Petersen says, “but the analog circuitry that’s built into a converter makes a huge difference on the sound, probably more so than the converter chip.”
Petersen cites cheap analog circuitry in an audio interface as contributing to the poor channel separation often associated with digital summing. While it’s not easy to verify the quality of the analog circuitry in an audio interface, looking at schematics can help. Petersen notes that low-cost interfaces will often follow the D/A converter chip with a stereo op amp, while his high-end gear follows the converter chip with two side-by-side mono amps on separate, non-touching circuit boards. “Are those cigarette pack-sized USB breakout boxes really treating the analog side of that signal properly?” Petersen asks. “Chance are usually no, they’re not.”

COMMON RESULTS

Everyone we asked had different opinions on the importance of summing circuits, but they all agreed on one important fact. The really critical part of sounding good when mixing with computers is the quality of the audio interface. If a sound card has poor D/A converters or poorly written drivers, then your software will sound bad. This is particularly important to remember as more and more DJs buy “all in one” controllers with built-in sound cards of sub-par quality. Wondering which sound card is right for you? Here are a few reviews of audio interfaces that we like, and another round up of “sound cards under $200″ is on the way.
Native Instruments Audio 8 Review
Echo AudioFire4 Reviewed
Further Complications.
As if this discussion were not already complicated enough, a really interesting fact popped up during the course of our investigation. Which operating system you use can actually play a part in the the latency and performance of audio coming out of your computer. It turns out that the Macintosh native audio drivers are much better suited for high-quality mixing than the native Windows drivers. Expect a full article on this subject in the coming weeks.

THE BOTTOM LINE

It’s safe to say that there is no real audio quality benefit to mixing with analog mixers instead of in the computer. In fact digital mixing offers greater flexibility in dealing with distortion and soft limiting. Assuming that you have good software, a proper audio interface and the common sense to set them both up properly, you should be able to get  a great-sounding mix either internally or externally. The real difference lies in each individual mixer and the quality of your audio interface.

Mix Techniques- Blending Swings

Mix Techniques- Blending Swings

Written by Ean Golden on March 15th, 2011 44 Comments

Today we follow up last month’s successful article on dancing with a more in-depth look at mixing various types of rhythmic styles together. In the video above, I go into detail on recognizing swing differences and then five techniques to smoothly and effectively switch between different rhythmic styles in both house and hip-hop. The mapping used in the video is our Midi-Fighter deckalized layout for Traktor Pro and is available as a free download here.

Traktor S4 Video Tutorial Pt 1/3: Mixer & Decks | Dubspot Blog

Traktor S4 Video Tutorial Pt 1/3: Mixer & Decks Dubspot Blog

The DJ Mothership: Traktor Pro 2 + NI’s Controllers – X1, S4, Maschine

The DJ Mothership: Traktor Pro 2 + NI’s Controllers – X1, S4, Maschine

10 Best Software for Visual Performance Artist

10 Best Software for Visual Performance Artist

Monday, December 14th, 2009

Share55

The task of selecting software for live audio-visual performances can become overwhelming, given the wide variety of solutions that are available. As is true of choosing anything, you need to understand what you need. You then take a look at what is available. Once again as with most other products, there are features offered that you may not comprehend completely. Your selection depends on your own skills and experience in the profession too. This can be especially tough for someone who is just starting out in this domain, or trying to get things done in such software for the first time, because you may not even be sure of what you need exactly. A good starting point is to take a look at what is available. And hence what are presented here are the best ones according to the professionals. So read on here to get the taste of some of the best VJ software that is available in the market today, including free ones right at the end.

VJamm

VJamm made by Camart Ltd., described by the software publisher as a “piano for sound and vision.” It comes in two primary variants, VJamm (£50) and the VJamm Pro (£299). The professional version carry a number of advanced features, including support for higher resolution clips – 2048×2048, (while VJamm supports only up to 720×576), support for more file types (“.mov”, “.swf”, “.3ds”), better media import methods (Live Text creation and SMS to screen) and the ability to record a particular composition for later use. The Pro edition also allows the purchaser access to VJamm’s development program, which in turn is one of the reasons why the latest versions have a lot of intuitive features – driven by inputs from VJ’s who have used VJamm in the field.
Providing a comprehensive audio-visual solution, VJamm is reputed among VJ’s as having one of the most accurate time-bases which make it highly suitable for professional presentations – clip stretching and other clip calisthenics. Other attractive features include a responsive cross-fade module, a range of on-the-fly clip controls (like soft edge luma key, opacity, etc.), support for up to 16 layers of video with individual controls for each, and more.
There are add-ons that can be downloaded such as VJamm Blueprint which is an intuitive tool for creating attractive visual improvisations on-the-fly. The great thing about the add-ons is that you don’t have to purchase them – they are available for free to download, and will work as long as you use the registration key that is provided during purchase of VJamm/Pro – the publisher’s promise “you pay only once.”
But VJamm is for Windows PCs only. Minimum requirements: 1GHz CPU, 256 MB RAM, Windows XP with an Open GL graphics card. Free demo version is also available.

AVmixer Pro

A good choice for starters, Neuromixer’s AVmixer Pro fits somewhere midway between VJamm and VJamm Pro. Although it does not have the support for as many live external video streams, it does come with a fairly complete set of features and so is an ideal solution for a beginning to intermediate level VJ. At $89, it is much cheaper than the VJamm Pro, and certainly a better buy than the VJamm. And unlike VJamm it is also available for Mac OS X.
The interface is attractive as well as intuitive. You can view the layers side by side for individual manipulation, and most of the functions, including a playback speed slider (range from 8% to 800%). With a little tinkering and some practice, you’ll very quickly find yourself sub-consciously rolling the mouse-over to the right place at the right time. It has a pretty standard list of advanced features including support for all major file types, MIDI support, support for FreeFrame effects and A/V mixing. The internal sequencer and the ability to assign specific audio channels to specific effects or to blends, makes it somewhat better VJamm.
As mentioned above, for Windows and Mac OSX users. Requires 512 MB RAM and QuickTime software installed prior to AVmixer Pro installation. Don’t forget to take advantage of the 30-day trial.

Resolume

For Mac as well as PC users, Resolume Avenue 3 is a great software for VJ`s, supporting up to 3 video layers. If you know to use Adobe Flash or use Flash for creating visuals then Resolume’s is for you. It can play Flash animations including AS2 and AS3 scripting. And when it comes to scripting you know how powerful it can be. It is probably a bit over-priced (€ 299.00) when compared to the VJamm Pro. But has the support for over a many external video streams and has relatively simplest user interface – overall it has a less cluttered feel. If you like to work on fewer layers, but would like far more video streams then this can be a good choice. Another important feature of this software is video mapping. If you have worked with much code based real-time graphics processing application like VVVV then you will know how hard it is to map a video on any physical object. But when it comes to Resolume with few clicks you can map the video onto any object or surface. Not only that you can even project on multiple surfaces or objects.
Although it does not have an SMS-to-Screen feature or an internal sequencer, it still comes with the ability to assign Audio input channels to effects and blends, as well as advanced LFO features – in addition to the standard LFO syncs to Audio/BPM (which available with VJamm Pro and AVmixer Pro), there is the ability to manually adjust the LFO wave and LFO assignment to mixing channels/effects.
You can watch the Resolume introduction video from Bill Holland at Gearwire. This software is available as a trial download as well as a 50% discount for an educational license. It requires at least 1GB of RAM with ATI Radeon 9600 or better OR NVIDIA GeForce FX 5200 or better.

ArkaOS Grand VJ

With an 8 Channel solution, ArkaOS Grand VJ (priced at €279) is comparable to the other heavyweights we have seen. So it is midway in terms of layering and video mixing capabilities. It does support most of the Automated Mixing and LFO capabilities, in a way similar to Resolume’s Avenue. Thus it clearly has the advantage over VJamm Pro in this regard, but falls behind in the numbers of layers (8 vs. 16). Whereas its advantage over Avenue is in these very layers (8 vs. 3) it does fall behind in the ability to support external video streams or clips (8 vs. over a hundred). So it really may boil down to your need for the additional layers at a slightly lower price.
Otherwise the feature-set is pretty standard and comparable to the other options: FreeFrame effects, MIDI support, and supports both Live Text Creation and SMS to Screen; like VJamm Pro you can save your compositions/performances.
Download and give it a try, but seriously consider paying the shipping for the boxed version because it comes with a 2.8GB DVD of clips from various VJ artists. 1 GB RAM recommended, XP/Mac OSX 10.4 or higher, NVIDIA GeForce FX 5200/ATI Radeon 9200/Intel GMA 950 or higher, and Quicktime+Flashplayer pre-installed.

VDMX

One of the most popular VJ software for the Mac platform is VDMX. Able to any number of input clips and up to 4 external video input sources, the VDMX comes fully-loaded with all the bells and whistles that are customarily expected from a professional VJ package. What makes VDMX stand out though is its insistence on a real-time paradigm, that grants access at runtime to advanced video/graphics manipulation functions. This access is provided by the user interface that displays the relevant graphs on the main screen itself – changing parameters can be done quickly on-the-fly and it’s as easy as adjusting the graphs with the mouse (which in turn will affect the internal engine; the rendering engine runs at runtime). This allows you the flexibility of trying out things spontaneously and be brilliantly creative. As the publisher’s point out “VDMX’s workflow is designed to let you treat your studio like an instrument.”
Indeed the interface, with its signal processing/video manipulation functions does give it a studio feel. The right half of the interface is completely dedicate to such “studio” functions. VDMX replaces the basic timeline paradigm that is so standard in VJ software today with their innovative take on the real-time studio feel. However, and it must be said, for the non-professional the interface can be a little over-whelming; and even for the professional there is a little bit of a cluttered feel (make sure you have a fairly large screen to make it easier for yourself) and it can take a while to get used to the new paradigm.
Another important feature of VDMX and advantage being a Mac OS X application is that it can import Quartz Composer Composition and control it in real-time using VDMX controls. It also can use Quartz Composer Patches as plug-ins. And when it comes to Quartz Composer, you how innovative and creative it can be.
At $300 it is a better deal than some of the software we have already discussed, and it has a variety of add-ons for streamlining the interface and the rendering pipeline. You can save a particular setup (i.e. interface & rendering-pipeline combination) and load whichever setup is most desirable for your current activity.
The trial version only has saving disabled. Requires 1GHz or better CPU, Mac OSX 10.5.0/QuickTime 7.0.4 up, 512 MB RAM, and PCI Express or Intel Graphics Processor with at least 64 MB VRAM and OpenGL.

Modul8

Another fairly popular Mac-only VJ solution is Modul8. Although priced in the same range as some of the software we have discussed here (@ €299; educational version @ €199), it does give you two licenses – possibly the freedom to maintain a Desktop and Laptop version at the same time. The interface does a lot by way of access to necessary functions and somehow feels a little bit better than a lot of competitors. This is partly because of the large preview screen as well as a fairly big (but not overwhelming) set of controls along the large left-pane.
Support for 10-layer video results in a rich composite output stream. Naturally each layer can be adjusted individually and in real-time, including the filters. And the rendering engine is optimized to run fairly fast – once again providing you with a flexibility that will enhance your creativity. Most features are pretty standard or are available as add-ons, such as sequencing and SMS-to-screen, the only thing that is lacking is the output options – For example there is no support DV/DMX output. Once you spend a little time bringing up the software up to your needs, you can be on your way. During that initial phase of discovery, another key thing is making the controls more flexible. Here, Modul8 easily outdoes everyone else, associating a keyboard stroke to a certain control is as easy as selecting the control (clicking) simultaneously with the desired key. In just a few minutes (or seconds, if you are one of those key-whizzes and know exactly what combinations will work for you) you can have an easy control set exactly customized to your needs.
The latest version if 2.6 and it comes with full FreeFrame and MIDI support (in fact MIDI signals can be configured to control the interface in a way similar to key-strokes) as well as for CoreImage filters; requires a 1GHz CPU, with Mac OSX 10.4.8/QuickTime 7.3.1 or higher, 1 GB RAM and AGP/PCI-Express Graphics with 64 MB VRAM; and the demo version is available here.

VVVV

A PC-based tool, VVVV is described by publisher as a “toolkit for real time video synthesis.” One of its most important features is its ability to simultaneously handle/manipulate a large number of objects. They call this feature spreading, i.e. spreading different parameter values across the large set of objects. This makes it easy for someone looking to use a large number of objects at the same time, and make behave in a certain way in real-time. VJs will appreciate the feeling of power that will inevitably strike them when they use “spreading.”
However, if you are a not so techie person, at first glance at the sheer number of options can overwhelm you. This is not to say that this is a big negative – the available functionality is certainly one of the things going for VVVV; but because of this large set of functions some time should have been spent improving interface design.
Professional VJs who like to tinker with every aspect of their A/V presentation will of course get over the interface hitches, and will love the Visual Programming Interface offered by VVVV, and the best thing is that prototyping and development takes place at runtime/real-time unlike other software that may implement these as separate modes or cycles. It is a great tool for 2D & 3D modeling and much more than VJ software, it is actually even a shoo-in for scientific applications. This makes it well-worth the €500 price tag, and best of all, VVVV is free for non-commercial use.
VVVV will handle as many as channels, layers and effects as your hardware can support, thus not applying any limitation of its own. While there is standard MIDI & FreeFrame support, VVVV does not have support for automated mixing.

MXWendler

Device+Context, the publishers, intend to offer MXWendler as a comprehensive solution for anyone looking for powerful A/V rendering and manipulation services. This naturally includes corporations in addition to artists. Apart from Stage Designer which is MXWendler’s software piece, the publishers also offer FXServer which is optimized to work well with the software. The hardware and software are pricey (hardware available at a €350 rental basis and an unquoted buying price; software is in excess of €850) and obviously the complete solution is not meant for a normal VJ. For the VJ there is the option of licensing the software on a monthly basis (@ 10% of list price) or take advantage of a 20% discount (only if you make less than 25000 €/year) to buy a Stage Designer Entropy Pack (3 options included; €850) or Stage Designer Eventdriver Pack (all options included; €2450). Go here for more pricing information.
MXWendler supports 10 layer video editing, 4 external video streams, with a maximum of 24 clips loadable simultaneously: each individually controllable. Its sleek interface makes it a joy to work with, and there is never any sense of clutter or incomprehension while working with it. Most features that come in other packages are standard on MXWendler and this is not surprising given how expensive it is (exceptions: no automated mixing, no SMS-to-Screen, no DV output, and most importantly NO FREEFRAME SUPPORT). The design philosophy is similar to VVVV, with a single real-time system and an output correction engine ensures that real-time multi-screen output is seamlessly achieved. You can view “555 KUBIK” installation to get a glimpse of its power.
The software is optimized for NVIDIA graphics cards (ATI is also supported). It works with 1 GB RAM (4GB recommended), 512 MB VRAM, and Mac OS X 10.4/Windows XP or later. There is no trial version.

Salvation

Harrison Digital Media’s Salvation is a Mac and PC based video processing and visual programming tool. It allows you to mix and manipulate as many video layers and effect as your hardware will allow. It comes with a GLSL pixel programming interface that can be used to create a host of unique effects – along with the multiple layers and individual manipulation of each, this promises practically limitless creativity on the part of the VJ. However, strangely, the output capabilities of that VJ will be somewhat limited because Salvation neither comes with DV output nor DMX. This is a severe limitation for software priced at $500.
The multiple windowing approach to the user interface, in turn, makes the software little confusing and cumbersome to use. But simplicity lies in its visual programming environment. Everything else is what you would expect from a professional software in this price bracket – FreeFrame & MIDI support, full LFO and automated mixing options, and SMS-to-Screen+Live-Text-Creation, among others.

They also have a video mixer called Salvation MX and it has 2 video channels (A and B; each can be manipulated) with the output being a composite. Salvation MX is available for an additional $49.99.
Salvation requires a 1GHz CPU, 256 MB RAM, OpenGL 2.0 Graphics card, and at least Windows XP. Trial version is available for 60 days.

Aestesis Electronika

The above mentioned options are great, but there are those of us out there who would just like to be able to do things for fun, as an enthusiast, without shelling out $ (or €). For their consideration there is the free Aestesis Electronika (for PC only) – although not in the big league by any means, it still offers a pretty decent set of features, including FreeFrame & MIDI support, the ability to select from up to 30 clips at a time, along with customization options. A rear view offers up the reverse side of your console/deck and there you can connect a number of A/V virtual wires to get the effects that you desire. If you’re a beginner, start here and work your way up. Check out the basic documentation and tips on the wiki.

How To Project On 3D Geometry

French | Russian

This article provides a manual approach to solving problems related to the following topics:

• Projecting on 3D Geometry
• Texturing real Architecture via Projections
• Projection/Video Mapping
• Undistoring Projections
• Augmented Reality
• Camera Matching/Resectioning/Calibration
• Pose Estimation

Note that due to the manual nature of the described approach your results will not be perfect, but can be quite good depending on your patience.

• Introduction
• Projection on a flat surface
• Projection on an arbitrary surface
• Manually matching a static virtual projector with a real projector
• Virtual replica of the real scene
• Flat textures aligned to the surface
• Lights and Shadows
• 3d illusion
• Virtual Camera's perspective is different to real world projector position
• Projecting on two sides of a house with one perspective
• 360° projection setup

Introduction

All the above mentioned topics deal with the same problem: Find the pose (position, orientation) of an object or the pose and lens characteristics (field of view, shift) of a camera viewing that object.
In cinematography with 3d visual effects the terms "match moving" or "camera tracking" are used to describe the problem of matching a real world scene with its virtual counterpart in order to mix them together seamlessly. For this task a list of software exists (boujou, 3d equalizer, ...) that makes the live of visual effects people easier.
For realtime computergraphics the ARToolKit provides a way that works for certain situations (see ARTK+Tracker? for a vvvv implementation) and companies like domeprojection offer projector autocalibration services. But as long as vvvv doesn't have nodes to patch something like this or even this the manual approach for static matching can still help to solve certain problems.

Projection on a flat surface

If your projection target is a flat surface, like a wall and your projector is at an arbitraty position, not exactly facing the part of the wall you want to project to, the projected image looks distorted. Using Homography (Transform 2d) you can easily pre-distort the image you project so that it appears undistorted on the surface.
Note: Like this you can always get a correctly looking projected image on a flat surface independent of the projectors position, orientation to the surface and its lens characteristics.

Projection on an arbitrary surface

Manually matching a static virtual projector with a real projector

When projecting onto an arbitratry 3d surface, no matter how the projector is positioned and oriented towards the surface the resulting image will mostly look distorted. Note though that there is one point from which the projected image looks perfectly aligned, that is: the position of the projector.
The photos below show a simple sculpture of 2 boxes with a windows-desktop projected flat on it. While in the left photo the projected image is distorted, in the right photo, taken from the projectors point of view, the projected image looks correctly.

Having seen the above you should now understand that the key to a correctly undistorted projection is the following:

A projection onto an arbitrary surface looks undistorted (ie. exactly as being projected) when seen from the point of view of the projector.

Also compare Distorted way-finding-system and 3D Painted Rooms.
So in order to achieve an undistorted look on an arbitrary surface you simply have to provide the projector with an image that depicts a view onto that surface from its own (the projectors) position. Or in other words: consider the real projector being a camera in virtual space, viewing a virtual replica of the real projection surface. If you project the image this virtual camera sees with the real projector, it would fit exactly and look undistorted on the real surface.
Note that this works perfectly only in theory, as chances are that your virtual scenes parameters will never exactly match your real world parameters, as some of them, like the projectors orientation, are hard to measure precisely. But never mind, there begins the manual tweaking of those parameters and with some patience and practice you'll get quite good results.

Virtual replica of the real scene

Creating a virtual copy of your real world setup includes three steps:

• define the origin for your real worlds coordinate system which you will match with your AxisAndGrid (EX9) ? within vvvv.

• create the target projection surface as a 3d model and place it correctly in your virtual scene regarding the coordinate systems origin: you can do this using an external 3d modelling tool or using vvvvs geometric primitivs. Anyway its best to have a mesh which you can finetune manually using PointEditor (3D Persistent) during the matching process.

• measure the position, orientation and lens-characteristics of the projector: use the Projector (EX9) module and specify all parameters this node offers. Look for sketches like the following in the projectors manual that provide you with the information for lens-shift and field of view of the lens.

At this point you have a basic setup where the virtual scene matches the real world. Take your time to fiddle with the parameters of the Projector (EX9) module or slightly adapt your mesh using the PointEditor (3D Persistent) module until you are satisfied with the overlapping of your virtual and real world scene.
ProjectionMapping_SimpleSetup.v4p (27.74 Kb)

Flat textures aligned to the surface

With this basic setup it is now simple to achieve an effect like in dotdotdot's Valcucine’s skyscraper project. Here it is the goal to project images in a way that they feel like a property of the target surface, ie. like they where a texture of the object. The textures should look exactly the same, from whatever position you view the surface/sculpture and certainly independent of the projectors position.

By rendering the 3d model from the same viewpoint and using the same lens-characteristics as the real projector has for the virtual camera, the resulting image will perfectly fit the projected surface. Any flat textures you give the 3d model (in the above example a simple black cross on white) will look undistorted on the real surface and therefore look and behave naturally like flat textures that look the same independent of the spectators point of view.

Lights and Shadows

Any shading (including lighting and shadowing) you apply to the virtual model is a surface oriented effect and as such behaves like a flat texture on the model. Effects like seen in our own Lightstrive and Pablo Valbuena's Augmented Sculpture Series can be realized just like this.

3d illusion

Fake 3d effects like seen in Pablo Valbuena's The Hague City Hall projection, AntiVJs Nuit Blanche Bruxelles projection and some of easyweb's projections contain an additional quirk you should be aware of.
While with the above setup those effects are quite easy to achieve, by simply adding 3d extensions to your virtual model, for a spectator they really only work from the one point in real world that corresponds to the virtual cameras position. Like this simple example shows, when viewed from about the projectors perspective the text seems to be extruded from the little box. When viewing the same projection from a completely different point, the illusion is gone.

But never mind, in many cases this isn't a big deal as our TV/cinema trained eyes and brains are quite tolerant with such visual challenges as long as our viewing position isn't too far off the actual position. Also compare John Beever's Pavement Drawings.

Virtual Camera's perspective is different to real world projector position

With some setups it is just not possible to have the real projector's position match the virtual camera's/projector's position:

• You may want the virtual scene to be viewed from a position where you cannot place the projector in real world
• You may want several real world projectors to share one virtual perspective.

In those situations you need 2 render-passes:

• the first pass renders the scene from the desired perspective
• the second pass renders the scene from the desired real world position of the projectors WHILE the result of the first pass is being projected onto the 3d models surface from the perspective of the virtual camera.

Examples:

Projecting on two sides of a house with one perspective

In this setup the idea was to project (among others) 3d-illusion effects onto the facade of the FH Schwäbisch Gmünd.

Knowing that those effects only work from one perspective a spectators view was chosen from which both sides of the facade can be seen. That is in the middle of the two real projectors to be used.
Here is a patch that demonstrates the basic setup:
ProjectionMapping_2ProjectorSetup.zip (12.11 Kb)

The above picture demonstrates that the 3d-illusion only works from the specified virtual camera/spectator position. For more results of the 5 days workshop in Schwäbisch Gmünd see here.

360° projection setup

In a 360° projection the virtual cameras all need to be placed in the same position, with only different rotations. While this is possible in a virtual scene you cannot put several physical projectors at exactly the same place in the real world.
For an extensive demo of a 360° projection setup, including soft-edging, see:
small 360 degree setup.zip (35.95 Kb)