As rama said, first I need to manage to get a "continuous" stream of data from the spdif videos. If/When I manage to get that working, then it should be possible (and probably not extremely hard) to send that data stream through the spdif port.
Of course it would be faster than decoding the stream in software, but not faster than ignoring spdif completely, and you happen to have a quad core, so you have cpu to spare.
Still: one thing at a time. There's no point thinking if passthrough is possible or faster, without a working spdif emulation.