Towards the connection to language, the concept has been published at the AAAI Conference in Stanford together with Srini Narayanan:
… we present a system of connected knowledge representations that is used to control a robot through instructions. As actions are a key component of in- structions and the robot’s behavior the representation of ac- tion is central in our approach. First, the system consists of a conceptual schema representation which provides a param- eter interface for action. Second, we present an intermediate representation of the temporal structure of action and show how this generic action structure can be mapped to detailed action controllers as well as language.
Citation: Schilling, M. and Narayanan, S. (2013), “Communicating with Executable Action Representations”. In Proceedings of AAAI Spring Symposium Series 2013, Stanford.