April 25 2000, Peter Dalgaard ============================== Including Tcl/Tk features in R ============================== Goal: Implement simple menus, dialogs, forms et al. Can be done at many possible levels -- try simplest: Wire Tcl event handling to read loop (like we currently do with X). Implement shallow binding: * R can call Tcl interpreter * Tcl scripts can call R. UNIX only (for now) A. Getting input ================ The first obstacle is to get Tcl/Tk to run windows and whatnot, simultaneously with R receiving and processing commands. The trouble is that both of them wants to take control over the event handling. Wiring Tcl into R's read loop ----------------------------- The R_ReadConsole function runs a loop, esssentially for (;;) { InputHandler *what = waitForActivity(); if(what != NULL) { if(what->fileDescriptor == fileno(stdin)) { if (UsingReadline) { rl_callback_read_char(); if (readline_eof) return 0; if (readline_gotaline) return 1; } else { if(fgets(buf, len, stdin) == NULL) return 0; else return 1; } } else what->handler((void*) NULL); } } With Tcl, one would like to replace the waitForActivity stuff (which is essentially a select() call) by a call to Tcl_DoOneEvent. It's not a big problem to handle file events in general, since we have an interface using Tcl_CreateFileHandler (which also works at the file descriptor level). For stdin activity, it should be possible to have the stdin handler just set a flag and return, as in [ static int stdin_activity; ] for (;;) { stdin_activity = 0; Tcl_DoOneEvent(); if(stdin_activity) { if (UsingReadline) { rl_callback_read_char(); if (readline_eof) return 0; if (readline_gotaline) return 1; } else { if(fgets(buf, len, stdin) == NULL) return 0; else return 1; } } } and the file event handler for stdin just has to set stdin_activity and return. No-stdio wiring --------------- Alternatively, one could set things up so that stdio is closed and the entire interaction takes place via a Tcl/Tk console ("R --gui=tk --no-stdio" or somesuch). In that case, the "buf" should be filled from Tcl along with the stdin_activity flag, and the loop could be just for (;;) { stdin_activity = 0; Tcl_DoOneEvent(); if(stdin_activity) return 1; } To that end one would need an interface function registered with the Tcl interpreter, something like this int R_send(ClientData clientData, Tcl_Interp *interp, int argc, char *argv[]) { strcpy(argv[1], buf); stdin_activity = 1; return TCL_OK; } and in the Tcl startup code .... Tcl_CreateCommand(Tcl_interp, "R_send", R_send, (ClientData) NULL, (Tcl_CmdDeleteProc *) NULL); .... The third way ------------- It is *almost* possible to bypass the console entirely and just have the Tcl side feed gobs of text to the parser/evaluator. This would be attractive in some ways: We could get rid of the line-by-line parsing and the need to discover and indicate when a line is syntactically continuable (the "+" prompt confuses some people badly) and we could move to a multiline send-and-recall structure like in interactive SAS, avoiding the quadratic complexity in recalling blocks of commands (to recall and send 8 successive lines, you need 64 up-arrow keypresses!). However, the snag is that some R code requests user input, either as confirmation indications (q(), to name an obvious case) or as data lines (scan()). So all of those cases need to be identified and replaced with alerts and popup entry windows. Wouldn't necessarily be a bad thing -- scan() on stdin is really not a very attractive technique -- but is sounds a bit daunting, so it should probably not be attempted as the first approach. Using threads ------------- All three above solutions have the problem that once R goes into "deep thought", no events are processed until it finishes its computation. In particular, there's no way of implementing a stop button to kill a runaway computation. This could be improved upon by having R run its computations in a separate thread of execution. This could be implemented using pthreads, by running R_ReplConsole in a separate thread, which the main thread could cancel, even inside C code. Commands could go from the GUI to the R_ReplConsole thread using message passing. As far as I can see both the Tcl interpreter and the R parser/evaluator must be treated as protected resources that can only be accessed by one process at a time. One does need to be careful not to generate deadlocks and the whole thing looks a bit tricky, but promising. (Re. thread killing, there is the notion of a cancel point, which needs to be handled with some care, in particular one has to make sure that a thread is not killed during garbage collection, since a half-done GC is likely to cause massive memory corruption!) B. Language bindings ==================== Of course the whole point of integrating R and Tcl/Tk rather than just building e.g. a GUI shell for R is that it should be possible to control the GUI from inside R and pass R data to the GUI. E.g. you'd want to setup an interface to lm() which can setup a listbox for the data= argument containing all available data frames. The first thing to consider is the mechanisms to pass commands from one language to the other. This is fairly easy to do, at least in the simplest variants. From C you can call Tcl_Eval() with any Tcl command, and of course you can write a trivial .C() interface to that from R. It is also possible to communicate at lower, more efficient levels. In the other direction, the option is basically to supply the Tcl interpreter with a command (R_eval, say) to send text strings to be parsed and evaluated by R. It might be possible to have a way to deal with preparsed code too. The next question is how to do the actual language bindings. Tcl has some aspects that make it difficult to map onto other languages. The structure hinges on "widget commands" where the whole window hierarchy is encoded in the command name. E.g. you configure a listbox entry in the bottom right subwindow using something like .entry.right.filebox configure -relief raised Obviously, this can get painful if the nesting level is deep. In Tcl itself, it is not all that bad because the language allows textual substitutions so you can do things like $w.filebox configure -relief raised but this doesn't transfer nicely to other languages. Instead, both Python and the STkLOS system embed the Tk commands in an object oriented structure. Below is first a piece of Tcl code and then a similar code in the tkinter Python embedding (dialog.tcl, and dialog.py from the respective distributions): --------Tcl------- proc tk_dialog {w title text bitmap default args} { global tkPriv tcl_platform # 1. Create the top-level window and divide it into top # and bottom parts. catch {destroy $w} toplevel $w -class Dialog wm title $w $title wm iconname $w Dialog wm protocol $w WM_DELETE_WINDOW { } # The following command means that the dialog won't be posted if # [winfo parent $w] is iconified, but it's really needed; otherwise # the dialog can become obscured by other windows in the application, # even though its grab keeps the rest of the application from being used. wm transient $w [winfo toplevel [winfo parent $w]] if {$tcl_platform(platform) == "macintosh"} { unsupported1 style $w dBoxProc } frame $w.bot frame $w.top if {$tcl_platform(platform) == "unix"} { $w.bot configure -relief raised -bd 1 $w.top configure -relief raised -bd 1 } pack $w.bot -side bottom -fill both pack $w.top -side top -fill both -expand 1 # 2. Fill the top part with bitmap and message (use the option # database for -wraplength so that it can be overridden by # the caller). option add *Dialog.msg.wrapLength 3i widgetDefault label $w.msg -justify left -text $text if {$tcl_platform(platform) == "macintosh"} { $w.msg configure -font system } else { $w.msg configure -font {Times 18} } pack $w.msg -in $w.top -side right -expand 1 -fill both -padx 3m -pady 3m if {$bitmap != ""} { if {($tcl_platform(platform) == "macintosh") && ($bitmap == "error")} { set bitmap "stop" } label $w.bitmap -bitmap $bitmap pack $w.bitmap -in $w.top -side left -padx 3m -pady 3m } # 3. Create a row of buttons at the bottom of the dialog. set i 0 foreach but $args { button $w.button$i -text $but -command "set tkPriv(button) $i" if {$i == $default} { $w.button$i configure -default active } else { $w.button$i configure -default normal } grid $w.button$i -in $w.bot -column $i -row 0 -sticky ew -padx 10 grid columnconfigure $w.bot $i # We boost the size of some Mac buttons for l&f if {$tcl_platform(platform) == "macintosh"} { set tmp [string tolower $but] if {($tmp == "ok") || ($tmp == "cancel")} { grid columnconfigure $w.bot $i -minsize [expr 59 + 20] } } incr i } # 4. Create a binding for on the dialog if there is a # default button. if {$default >= 0} { bind $w " $w.button$default configure -state active -relief sunken update idletasks after 100 set tkPriv(button) $default " } ... } ------End Tcl ------- ------Python--------- def dialog(master, title, text, bitmap, default, *args): # 1. Create the top-level window and divide it into top # and bottom parts. w = Toplevel(master, class_='Dialog') w.title(title) w.iconname('Dialog') top = Frame(w, relief=RAISED, borderwidth=1) top.pack(side=TOP, fill=BOTH) bot = Frame(w, relief=RAISED, borderwidth=1) bot.pack(side=BOTTOM, fill=BOTH) # 2. Fill the top part with the bitmap and message. msg = Message(top, width='3i', text=text, font='-Adobe-Times-Medium-R-Normal-*-180-*') msg.pack(side=RIGHT, expand=1, fill=BOTH, padx='3m', pady='3m') if bitmap: bm = Label(top, bitmap=bitmap) bm.pack(side=LEFT, padx='3m', pady='3m') # 3. Create a row of buttons at the bottom of the dialog. var = IntVar() buttons = [] i = 0 for but in args: b = Button(bot, text=but, command=lambda v=var,i=i: v.set(i)) buttons.append(b) if i == default: bd = Frame(bot, relief=SUNKEN, borderwidth=1) bd.pack(side=LEFT, expand=1, padx='3m', pady='2m') b.lift() b.pack (in_=bd, side=LEFT, padx='2m', pady='2m', ipadx='2m', ipady='1m') else: b.pack (side=LEFT, expand=1, padx='3m', pady='3m', ipadx='2m', ipady='1m') i = i+1 # 4. Set up a binding for , if there's a default, # set a grab, and claim the focus too. if default >= 0: w.bind('', lambda e, b=buttons[default], v=var, i=default: (b.flash(), v.set(i))) oldFocus = w.focus_get() w.grab_set() w.focus_set() ... ----- End Python ---- Notice how Tcl code like label $w.msg -justify left -text $text $w.msg configure -font {Times 18} gets turned into Python msg = Message(top, width='3i', text=text, font='-Adobe-Times-Medium-R-Normal-*-180-*') Never mind that the separate configure step is omitted and that there appears to be some minor differences in the parameter settings. The important thing is that in Python the Message function creates an object which can be assigned to a variable, and whose creation is based on the parent window. The Tk name of the widget never emerges. (It is of course present as an ID fields somewhere inside msg). Note also that Python has the classic OOP style where objects carry their own method slots, hence the w.title(title) bd.pack(side=LEFT, expand=1, padx='3m', pady='2m') and soforth. For the R and S languages, one wouldn't do it precisely that way, more likely one would use generic functions and maybe also some of the other syntactic niceties. One could consider implementing the w.title(title) as title(w) <- title with w an object class "tkTopLevel" and "title<-" a generic function dispatching to "title<-.tkTopLevel" which would do something pretty close to "title<-.tkTopLevel" <- function(w, value) Tk_Eval(paste("wm title", ID(w), value)) [This might be a bit of an overkill, perhaps it might suffice to do it with straight function calls. i.e. title(w) # returns the title title(w, value) # sets the title to value ] So, just to have a sketch of how things might work, here's how I picture a dialog.R: ---------------- # # NOTE: This is not working code! It is not even a proposal for # what should eventually become working code... # dialog <- function(master=TkRoot, title="Dialog", text, args, bitmap=NULL, default=0) { # 1. Create the top-level window and divide it into top # and bottom parts. w <- Toplevel(master, class="Dialog") title(w) <- title iconname(w) <- "Dialog" top <- Frame(w, relief=RAISED, borderwidth=1) bot <- Frame(w, relief=RAISED, borderwidth=1) pack(top, side=TOP, fill=BOTH) pack(bot, side=BOTTOM, fill=BOTH) # 2. Fill the top part with the bitmap and message. msg <- Message(top, width="3i", text=text, font="-Adobe-Times-Medium-R-Normal-*-180-*") pack(msg, side=RIGHT, expand=1, fill=BOTH, padx="3m", pady="3m") if (!is.null(bitmap)) pack(Label(top, bitmap=bitmap), side=LEFT, padx="3m", pady="3m") # 3. Create a row of buttons at the bottom of the dialog. buttons = vector("list", length(args)) var <- default i <- 1 for (but in args) { b <- Button(bot, text=but, command=eval(substitute(function() var <<- i))) buttons[i] <- b if (i == default) { bd <- Frame(bot, relief=SUNKEN, borderwidth=1) pack(bd, side=LEFT, expand=1, padx="3m", pady="2m") lift(b) # not quite sure what this is supposed to do # Tcl version has # $w.button$i configure -default active pack(b, in=bd, side=LEFT, padx="2m", pady="2m", ipadx="2m", ipady="1m") } else { pack(b, side=LEFT, expand=1, padx="3m", pady="3m", ipadx="2m", ipady="1m") } i <- i+1 } # 4. Set up a binding for , if there's a default, # set a grab, and claim the focus too. if (default >= 0) bind(w, "", function() { flash(b[default]) ; var <<- default)}) oldFocus <- focus.get(w) grab.set(w) focus.set(w) ... } ----------- This was a pretty brutal near-copying of the Python code, but it would seem to be possible to make it work. A number of points to watch out for: 1) The buttons need to be able to share the variable "var" for their actions, so the commands are defined as function closures. This gets a bit clumsy in the button definition loop. An alternate possibility would be to define the commands as calls, and pass the environment in which to evaluate them (defaulting to the parent of the creation call). Then one could have b <- Button(bot, text=but, command=substitute(var <- i, list(i=i))) or maybe e <- new.env() evalq(var <- default, e) b <- Button(bot, text=but, command=substitute(var <- i), env=e) 2) Some functions correspond to "widget commands" in Tcl, others to direct Tcl commands ("pack") or variant Tcl commands ("wm title"). Actually, this example doesn't really have widget commands, but we could have had configure(b[default], state=active, relief=sunken) I think it is OK to have this mixture of generic and non-generic functions, but we might want to either rename the latter to something like pack.Tk, just to avoid potential name conflicts. In fact, the "grid" geometry manager already has problems... 3) widget commands like configure can be done in more or less elaborate ways. The most simple way is to just have a single configure.TkObject defined as something like Tk_Eval(paste(ID(x), configure, options(...))) but not all widgets accept the "configure" command, and we'd be letting Tcl point out the user errors. Alternatively, one could set up a class hierarchy (in fact this has already been done for STkLOS) and only have a configure method for objects inheriting from the appropriate class. 4) Naming conventions for Tcl sub commands, e.g. "pack configure". I'd suggest something in the style of pack.config() for that. Python has this as Pack.config, a class method if I understand correctly. 5) Timed-reaction commands like "after" need special care since we're never going to catch them if the Tk loop isn't kept running. Both in threaded and unthreaded implementations there would seem to be a potential for deadlocking the system. -------------- May 12 2000, Peter Dalgaard Embedding X11 Windows in Tk applications ======================================== The following sequence of steps would seem to work to create menu-fied X11 graphics window: 1) Create a toplevel widget with a menu and the -container switch set. Notice that the menu can not be a subwindow of the toplevel widget. (A container window cannot have subwindows.) The Tcl commands to do that would be something close to this menu .menu .menu add command -label "Print" toplevel .top -menu .menu -container 1 2) Get the Window ID of .top . This can be done with the "winfo id" construct: winfo id .top This returns a hex coded string, e.g. 0x8400007. 3) In X11_Open, we have if ((xd->window = XCreateWindow( display, rootwin, .... replace rootwin with an int32 version of the hex string from 2) and, bingo... For an actual implementation, one also needs to figure out how to pass the information around, with special attention to the issues of multiple devices (e.g. if one adds a "print" button, it had better print this window and not one of the others...) Possibly, the best idea is to pass a new argument to X11DeviceDriver containing the hex encoded parent. Then the embellishments could all be coded in R. Similar ideas apply to the data editor, but first we'd need to prevent the data editor from stalling the Tcl event loop...