Integrating CDP into other software

[loganmcbroom]

I'm working on a sort of front-end and running the CDP executables as child processes, but I'd like to be getting those processes output streams and apparently that can only be done on Windows using pipes which I don't understand at all. Being able to call the functions directly would also allow some additional flexibility.

Does anyone have experience or advice on this sort of integration, or on using Windows pipes?

[Xenakios]

The C++ frontends I've worked on, use the JUCE library, which indeed implements child processes with the help of pipes on Windows. There's by the way a problem in JUCE's implementation which makes the ChildProcess hang with some of the CDP programs. If you happen to be using C++ and decide to use JUCE too, I can give you the one liner fix. (The JUCE developer didn't consider this important enough to merge the fix into the official JUCE source code.)

In any case, you should really look into some cross platform library/framework that deals with this kind of stuff for you. Using the win32 API directly is a massive headache and obviously not cross platform either. I would recommend looking into Qt or JUCE. (JUCE is my current preferred choice, it also has a history of being widely used for audio applications/plugins.)

[loganmcbroom]

Well that's fantastic news because I am using C++ and I am using JUCE. This is the first thing I've used JUCE for and I didn't realize it had something built in for handling child processes... Really a huge relief as the windows api has indeed been a massive headache. Just managed to get pipes working but I am more than happy to tear it back down.

I can't imagine a more satisfying reply, I'll take that fix if you don't mind.

[Xenakios]

Well that's fantastic news because I am using C++ and I am using JUCE. This is the first thing I've used JUCE for and I didn't realize it had something built in for handling child processes... Really a huge relief as the windows api has indeed been a massive headache. Just managed to get pipes working but I am more than happy to tear it back down.

I can't imagine a more satisfying reply, I'll take that fix if you don't mind.

In the file juce_win32_Threads.cpp, there's the line :

if (CreatePipe (&readPipe, &writePipe, &securityAtts, 0)

If that is left like that, it may cause for example CDP's pvoc.exe to hang with long input files. Changing it to :

if (CreatePipe (&readPipe, &writePipe, &securityAtts, 1048576)

Should fix that problem. The number 1048576 is of course a bit arbitrary, it may be too big for most uses of the ChildProcess and may still be too small for some cases. But so far that has worked for me. (The number is the buffer size used by the pipe. The number 0 makes the buffer size be some undefined default size which doesn't play well with some of the CDP programs.)

[loganmcbroom]

Alright, seems like everything is running correctly. I'll post the project once I've polished it a bit.

Thanks for the help.

[clown]

Could we have some screenshots, logan?
I'm anxious 

[loganmcbroom]

Well, there's not a huge amount a screenshot will show but I'll post what I have. The concept I'm working on only has a small gui for getting files in, managing processing threads (working on this now, I've never dealt with threads so it's taking some extra time) and seeing the output. The actual descriptions for how the audio should be processed are in a Lua script which is invoking c functions which call CDP modules.

One of the things I think is missing from most front-ends is pure programmatic control. Being able to take a file and iteratively process it 1000 times, or applying a number of processes in some algorithmic way is hard to do without access to the control flow structures of a plain old programming language.

There are a few important bits still missing but it is in working condition. I'll post a proper exe download once threads are done-ish. Still need to add drag and drop to reorder list items, running scripts in threads, being able to start a cdp module without blocking the current thread, and helper functions for individual processes deemed important enough. Currently the only helper function is normalize although that may not be on github.

Source and a screenshot:

https://github.com/loganmcbroom/Altar
http://i.imgur.com/PPhinWM.png?1

I haven't used github before so if I've committed some faux pas let me know. I'm also fairly inexperienced so if you see any programming goofs point them out.

[Xenakios]

running scripts in threads, being able to start a cdp module without blocking the current thread

This probably isn't really needed. The new CDP frontend I worked on myself doesn't use threads for running the CDP programs but it yet can keep the GUI thread running and also leverage multicore CPUs for running in parallel. The trick is in letting the JUCE ChildProcesses run without waiting for them to finish in the current thread. You will need to keep the ChildProcess instances stored as member variables of some suitable class. (Or have them as global variables.) Then use Juce's timer to periodically check if the ChildProcess or processes have finished.

[loganmcbroom]

The separate threads wouldn't be for individual processes, but for the entire series of processes described by a script. That way you can run several long scripts at a time. Splitting to a new thread also takes away the need for polling running processes.

It leaves a problem as well though. I would guess in your front-end for each process you check to see when all the necessary files have been created and then process them. It's hard to do that in a script. Previously the only thing that came to mind to allow running multiple processes in parallel within a script was a function for starting the child process without blocking the thread and some functions for managing things.

Idea though: Rather than executing anything as the script runs, the script could patch together something more like what you have internally. That would allow similar parallelism. I am feeling way too lazy to try that atm but maybe if I run out of other things to do.

Thanks for the suggestions. I saw you mention taking a break on your front-end, still on pause?

[loganmcbroom]

Running into a problem with recsf. Using Juce::ChildProcess you can't send input to the spawned process so it can't be started (it requires a keypress to begin recording).

I could use juce for the recording in that function but I'd rather not. Thoughts?

[Xenakios]

I could use juce for the recording in that function but I'd rather not. Thoughts?

Why not? Implementing recording with Juce shouldn't be a big deal, maybe a few dozen lines of code assuming you already have the realtime audio stuff anyway for playback. If you need to add the realtime audio things first, that will require a few dozen lines of code.

[loganmcbroom]

Well I'd like to keep the cdp interface consistent but I don't want to handle all the flags recsf has. Looking at those flags though I realized there is a -i flag which solves this problem by starting the recording immediately.