Last updated: 2015-02-10 · 28 commits

Xwidgets

This is an experimental branch to enable embedding of GTK widgets inside an Emacs window. The Emacs abstraction is called an Xwidget, for eXternal widget, and also in reference to the Xembed protocoll.

There is a demo file called xwidget-test.el which shows some of the possibilities. There are some screnshots at the emacswiki.

Currently its possible to insert buttons, sliders, xembed widgets, and webkit in the buffer. It works similar to the support for images in Emacs. Adding more types of widgets should be fairly straightforward, but will require adapter code for each type.

A difference from images is that xwidgets live their own life. You create them with an api, get a reference, and tie them to a particular buffer with a display spec.

Each xwidget can have several views. In MVC terms, an xwidget is the model, and an xwidget-view is a view of the xwidget in a particular Emacs window.

The xwidget code attempts to keep the visual appearance of the views in sync with through an Observer pattern implementation. This is necessary to support the Emacs window paradigm.

building

These instructions are for a user, as opposed to a comitter, checking out the source and building.

git clone git://git.sv.gnu.org/emacs.git  
cd emacs
git checkout xwidget
# if you want to compile with debug
export CFLAGS=" -g"
./autogen.sh # if we forgot to commit the generated configure
./configure --enable-gcc-warnings --with-xwidgets --with-x-toolkit=gtk3
make -j4
# running in gdb is useful for bug reports
gdb -ex run src/emacs

try it out

If you have GTK3 and gtk-webkit installed, you should be able to start the embedded webkit browser now:

M-X xwidget-webkit-browse-url

If that didnt work out try the minimal demonstration instead:

(load-library "xwidget-test") (xwidget-demo-a-button)

It looks unimpressive, but it's a gtk button inside an Emacs buffer!

webkit hints

If you got webkit working, great! Please note, though, that the current support is far from a full fledged browser. My focus is on delivering a component that can be used to build a full emacs based browser on. Since I implement a browse-url function you can get quite far just by:

(setq browse-url-browser-function 'xwidget-webkit-browse-url)

then all Emacs browser interface systems work to a degree. heres stuff I use currenly

etc..

I'll add more examples as I go along.

However theres lots of support missing, see TODO list for information. Briefly:

unlikely to be fixed. Just a heads up.

Stability

Beginning with Summer 2011 I am now able to use Xwidget Emacs as my primary Emacs. That is good for the project and the stability of the code.

At the time of writing I have 24 hour Emacs uptime with several embedded webkit browsers, Gnus, org-mode, tramp, etc. etc.

That said, there are still many improvements that needs to be done, particularily in memory management. Expect xwidget emacs to leak heavily for now.

timeline for inclusion in trunk

The xwidget-mvp branch was merged to emacs-25. This branch is stripped down to only include the webkit widget.

reporting bugs

Emacs xwidgets uses the same tracker as mainline emacs, but a different package. To report a bug: M-x report-xwidget-bug

browse bugs: http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs-xwidgets

build debug

maintainer proceedures

In the below instructions, replace jave with your Savannah handle.

bzr init-repo emacs2014
cd emacs2014
bzr branch bzr+ssh://jave@bzr.savannah.gnu.org/emacs/trunk trunk
bzr branch bzr+ssh://jave@bzr.savannah.gnu.org/emacs/xwidget

cd xwidget

./autogen.sh
export CFLAGS=" -g"
./configure --with-xwidgets --with-x-toolkit=gtk3
make -j4

resolving conflicts, for instance: bzr –take-other .gitignore

bzr commit -m"upstream" bzr push :parent

Repository notes

Only the Savannah repository can be considered canonical.

Emacs upstream has switched to git, and the the xwidget repo has also been switched over.

There are several clones of the main repo lying about. Please ignore them.

Thanks

emacs-devel@gnu.org. There are very helpful people there. When I started the xwidget project I had no clue about the Emacs internals.

Screenshots

file:EmacsXembedScreenshot.png file:WebkitElnodeScreenshot.png

Brief overview of how xwidgets work

Xwidgets work in one way like images in Emacs. You bind a display spec very similar to an image display spec to buffer contents. The display engine will notice the display spec and try to display the xwidget there. The display engine prepares space at the right place for the xwidget and so on for free, as long as we provide proper sizes and so on back to the redisplay engine.

Issues

The problem is that Emacs cant actually draw the widgets, as it can with images. Emacs must notify GTK about where the widgets should be, and how they should be clipped and so on, and this information must be given to GTK synchronous with Emacs display changes. Ok, so why is that difficult then?

placement and clipping

the entire emacs frame is a gtk window. we use the fixed layout manager to place xwidgets on the frame. coordinates are supplied by the emacs display engine. widgets are placed inside an intermediate window, called the widgetwindow. the widgetwindows are placed on the emacs frame.

this way was chosen to simplify clipping of the widgets against emacs window borders.

different strategies

Integrating toolkit widgets(gtk in this case) and the emacs display engine is more difficult than your plain average gui application, and different strategies has been tested and will continue to be tested.

There was a distinction between live xwidgets and phantom xwidgets, previous to the change to MVC.

the drawback of that aproach was that the gtk toolkit is admirably lazy and doesnt draw the widget if its not actualy shown, meaning that the snapshots for the phantoms will show garbage.

this has the primary advantage that the snapshot is always available, and enables the possibility of more eye-candy like drawing live and phantom widgets in different colors.

the drawback is that its our own responsibility to handle drawing, which puts more of the display optimization burden on us.

this is aproach worked so-so.

Testing

;;test like: ;; cd /path/to/xwidgets-emacs-dir ;; make all&& src/emacs -q –eval "(progn (load \"`pwd`/lisp/xwidget-test.el\") (xwidget-demo-basic))"

MVC and Xembedd

The MVC approach appears to be at least in principle robust for plain gtk widgets. For the interesting case of gtk sockets which implements an xembed host widget that allows for embedding other applications inside an Emacs window, the story gets more complex.

The problem is that xembed is designed to plug an application window inside a socket and thats it. You can't move a plug between sockets. I tried numerous hacks to get around this but there is nothing that works really well.

Therefore the Emacs part of the code will only expose well-defined interfaces. cooperating applications will be able to use the interface in a well defined manner. The problem is that there is no known xembeddable application that implement the needed type of functionality, which is allowing for creating new windows on the fly that plug into new sockets.

Therefore I will attempt to provide an external application that wraps another application and through hacks attempts to provide the needed multi view xembed function. That way Emacs is sane and the insanity contained.

This app will work by providing a socket that an app plugs into. The socket window is copied efficiently by means of composition to a number of other windows, that are then plugged into the different Emacs sockets.

old notes from xdrawxwidgetglyphstring

BUG it seems this method for some reason is called with bad s->x and s->y sometimes. When this happens the xwidget doesnt move on screen as it should. This might be because of xscrollrun. Emacs decides to scroll the screen by blitting sometimes. then emacs doesnt try to actualy call the paint routines, which means this here code will never run so the xwidget wont know it has been moved.

Solved temporarily by never optimizing in trywindowreusingcurrentmatrix().

BUG the phantoming code doesnt work very well when the live xwidget is off screen. you will get weirdo display artefacts. Composition ought to solve this, since that means the live window is always available in an off-screen buffer. My current attempt at composition doesnt work properly however.

//allocation debugging. the correct values cant be expected to show upp immediately, but eventually they should get to be ok // this is because we dont know when the container gets around to do layout //GtkAllocation galloc; //gtkwidgetgetallocation(GTKWIDGET (xv->widgetwindow), &galloc); //printf("allocation %d %d , %d %d", galloc.x,galloc.y,galloc.width,galloc.height);

old notes about the old live/phantom scheme

//TODO: // 1) always draw live xwidget in selected window // (2) if there were no live instances of the xwidget in selected window, also draw it live) // 3) if there was a live xwidget previously, now phantom it.

else { //ok, we are painting the xwidgets in non-selected window, so draw a phantom //printf("draw phantom xwidget at:%d %d",x,y); //xwidgetcompositedrawphantom (xw, x, y, clipx, clipy); //TODO MVC there will be very few cases of phantoming }

atm this works as follows: only check if xwidgets are displayed in the "selected window". if not, hide them or phantom them.

this means valid cases like xwidgets being displayed only once in non-selected windows, does not work well. they should also be visible in that case not phantomed.

ToDo:s

TODO webkit crash

[2013-04-13 Sat] seems to crash a lot on http://www.dilbert.com Not always, but enough to be annoying.

TODO optimize drawing off large offscreen widgets

Currently I just allocate as large an area as the offscreen webkit widget desires. This works well most of the time. But a HTML page might in principle be of infinite height so there are cases where this doesn't work too well.

Heres a proposed strategy:

DONE again a trace

[2011-08-23 Tue] the hunch is that since I still hand-wave the view storage the array can get out of synchronous. so maybe switching to a lisp structure will help as it did for the model. Anyway, doesnt happen at all often.

the trace

(gdb) bt
#0  0x0000000000685304 in xwidget_touch (xv=0x0) at xwidget.c:1225
#1  0x00000000006853e7 in xwidget_end_redisplay (w=0x11b42ca0, matrix=
    0xff9bf40) at xwidget.c:1272
#2  0x000000000041cc31 in update_window (w=0x11b42ca0, force_p=0)
    at dispnew.c:3705
#3  0x000000000041c0e5 in update_window_tree (w=0x11b42ca0, force_p=0)
    at dispnew.c:3331
#4  0x000000000041be8b in update_frame (f=0x1682a50, force_p=0, 
    inhibit_hairy_id_p=0) at dispnew.c:3258
#5  0x000000000045066f in redisplay_internal () at xdisp.c:12931
#6  0x000000000044e210 in redisplay () at xdisp.c:12110
#7  0x0000000000567e65 in read_char (commandflag=1, nmaps=7, maps=
    0x7fffffffc040, prev_event=12708226, used_mouse_menu=0x7fffffffc254, 
    end_time=0x0) at keyboard.c:2447
#8  0x000000000057613c in read_key_sequence (keybuf=0x7fffffffc4a0, bufsize=
    30, prompt=12708226, dont_downcase_last=0, can_return_switch_frame=1, 
    fix_current_buffer=1) at keyboard.c:9299
#9  0x0000000000565d45 in command_loop_1 () at keyboard.c:1448
#10 0x0000000000601008 in internal_condition_case (bfun=
    0x565962 <command_loop_1>, handlers=12760466, hfun=0x565259 <cmd_error>)
    at eval.c:1490
#11 0x0000000000565659 in command_loop_2 (ignore=12708226) at keyboard.c:1159
#12 0x0000000000600992 in internal_catch (tag=12873826, func=
---Type <return> to continue, or q <return> to quit---
    0x565633 <command_loop_2>, arg=12708226) at eval.c:1247
#13 0x00000000005655bd in command_loop () at keyboard.c:1124
#14 0x0000000000564da7 in recursive_edit_1 () at keyboard.c:759
#15 0x0000000000564f43 in Frecursive_edit () at keyboard.c:823
#16 0x000000000060444f in Ffuncall (nargs=1, args=0x7fffffffca20)
    at eval.c:2986
#17 0x00000000006507f8 in exec_byte_code (bytestr=145172929, vector=145179445, 
    maxdepth=116, args_template=12708226, nargs=0, args=0x0) at bytecode.c:785
#18 0x0000000000604eec in funcall_lambda (fun=140575909, nargs=2, arg_vector=
    0x7fffffffcfe8) at eval.c:3220
#19 0x000000000060467e in Ffuncall (nargs=3, args=0x7fffffffcfe0)
    at eval.c:3038
#20 0x00000000006035fc in Fapply (nargs=2, args=0x7fffffffd0b0) at eval.c:2494
#21 0x0000000000603b43 in apply1 (fn=12874242, arg=301666310) at eval.c:2732
#22 0x00000000005feb25 in call_debugger (arg=301666310) at eval.c:220
#23 0x0000000000601ca9 in maybe_call_debugger (conditions=9431542, sig=
    12761282, data=301666742) at eval.c:1893
#24 0x0000000000601785 in Fsignal (error_symbol=12761282, data=301666742)
    at eval.c:1714
#25 0x0000000000601898 in xsignal (error_symbol=12761282, data=301666742)
    at eval.c:1749
#26 0x0000000000601926 in xsignal2 (error_symbol=12761282, arg1=102756373, 
    arg2=0) at eval.c:1770
---Type <return> to continue, or q <return> to quit---
#27 0x0000000000604d6e in funcall_lambda (fun=102756373, nargs=0, arg_vector=
    0x7fffffffd398) at eval.c:3189
#28 0x000000000060467e in Ffuncall (nargs=1, args=0x7fffffffd390)
    at eval.c:3038
#29 0x00000000006507f8 in exec_byte_code (bytestr=54783137, vector=109656229, 
    maxdepth=12, args_template=12708226, nargs=0, args=0x0) at bytecode.c:785
#30 0x0000000000604eec in funcall_lambda (fun=109656517, nargs=0, arg_vector=
    0x7fffffffd890) at eval.c:3220
#31 0x000000000060467e in Ffuncall (nargs=1, args=0x7fffffffd888)
    at eval.c:3038
#32 0x0000000000603b08 in apply1 (fn=109656517, arg=12708226) at eval.c:2725
#33 0x00000000005fc8c9 in Fcall_interactively (function=109656517, record_flag=
    12708226, keys=12754549) at callint.c:379
#34 0x00000000006044c2 in Ffuncall (nargs=4, args=0x7fffffffdc60)
    at eval.c:2996
#35 0x0000000000603c57 in call3 (fn=12893554, arg1=109656517, arg2=12708226, 
    arg3=12708226) at eval.c:2789
#36 0x00000000005784cd in Fcommand_execute (cmd=109656517, record_flag=
    12708226, keys=12708226, special=12708226) at keyboard.c:10290
#37 0x00000000005661fb in command_loop_1 () at keyboard.c:1575
#38 0x0000000000601008 in internal_condition_case (bfun=
    0x565962 <command_loop_1>, handlers=12760466, hfun=0x565259 <cmd_error>)
    at eval.c:1490
---Type <return> to continue, or q <return> to quit---
#39 0x0000000000565659 in command_loop_2 (ignore=12708226) at keyboard.c:1159
#40 0x0000000000600992 in internal_catch (tag=12756258, func=
    0x565633 <command_loop_2>, arg=12708226) at eval.c:1247
#41 0x000000000056560c in command_loop () at keyboard.c:1138
#42 0x0000000000564da7 in recursive_edit_1 () at keyboard.c:759
#43 0x0000000000564f43 in Frecursive_edit () at keyboard.c:823
#44 0x0000000000563052 in main (argc=1, argv=0x7fffffffe678) at emacs.c:1711

Lisp Backtrace:
"recursive-edit" (0xffffca28)
"debug" (0xffffcfe8)
"image-bol" (0xffffd398)
0x68939c0 PVEC_COMPILED
"call-interactively" (0xffffdc68)
(gdb)

DONE new annoying trace

maybe related to scroll inhibiting or cursor inhibiting code. It appears actually to be related to GLYPHDEBUG=1. this flag is no longer needed.

the trace



Breakpoint 1, abort () at emacs.c:383
383       kill (getpid (), SIGABRT);
Missing separate debuginfos, use: debuginfo-install hunspell-1.2.15-2.fc15.x86_64 nss-mdns-0.10-9.fc15.x86_64
(gdb) 
(gdb) 
(gdb) bt
#0  abort () at emacs.c:383
#1  0x0000000000418f01 in matrix_row (matrix=0xac29400, row=-1)
    at dispnew.c:1477
#2  0x000000000046e113 in draw_glyphs (w=0x18235c0, x=198, row=0xa3af100, area=
    TEXT_AREA, start=17, end=18, hl=DRAW_CURSOR, overlaps=0) at xdisp.c:22550
#3  0x000000000047869f in draw_phys_cursor_glyph (w=0x18235c0, row=0xa3af100, 
    hl=DRAW_CURSOR) at xdisp.c:24882
#4  0x00000000005083bb in x_draw_window_cursor (w=0x18235c0, glyph_row=
    0xa3af100, x=180, y=361, cursor_type=0, cursor_width=1, on_p=1, active_p=1)
    at xterm.c:7440
#5  0x00000000004790cd in display_and_set_cursor (w=0x18235c0, on=1, hpos=17, 
    vpos=19, x=180, y=361) at xdisp.c:25098
#6  0x00000000004fa31f in x_update_window_end (w=0x18235c0, cursor_on_p=1, 
    mouse_face_overwritten_p=0) at xterm.c:644
#7  0x000000000041ccb9 in update_window (w=0x18235c0, force_p=0)
    at dispnew.c:3694
#8  0x000000000041c165 in update_window_tree (w=0x18235c0, force_p=0)
    at dispnew.c:3331
#9  0x000000000041beee in update_frame (f=0x1658460, force_p=0, 
    inhibit_hairy_id_p=0) at dispnew.c:3258
#10 0x0000000000450a2e in redisplay_internal () at xdisp.c:12983
#11 0x000000000044e2a6 in redisplay () at xdisp.c:12099
#12 0x000000000056a60d in read_char (commandflag=1, nmaps=6, maps=

DONE allow xwidgets to report their size

now we just hard code sizes. but webkit widgets for instance can report sizes that suit the content. support that.

DONE BUG xwidget view ghosts

(havent seen this in quite a while)

now theres 2 webkit views

now theres 2 views but one is a ghost! one should have been deleted when its window died but that didnt work for some reason here.

the ghost goes away because we killed explicitly but this is just a workaround.

xwidgetviewdeleteallinwindow(w); in delete-window-internal is not sufficient. delete-other-windows-internal deleteallsubwindows unshowbuffer

Added cleanup those window configuration hook which works in practice but feels kludgy.

code looks like this

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; (defun xwidget-cleanup () "Delete zombie xwidgets." ;;its still pretty easy to trigger bugs with xwidgets. ;;this function tries to implement a workaround (interactive) (xwidget-delete-zombies) ;;kill xviews who should have been deleted but stull linger (redraw-display);;redraw display otherwise ghost of zombies will remain to haunt the screen )

;;this is a workaround because I cant find the right place to put it in C ;;seems to work well in practice though (add-hook 'window-configuration-change-hook 'xwidget-cleanup)

but it ought rather to work like this

xwidget-delete-zombies should be called from C after window configuration has changed but before redisplay. redisplay should not be called.

DONE BUG annoying backtrace

(this no longer seems to happen even under heavy usage. seems merging from trunk helped. lots were happening in redisplay at this time in trunk.)

sadly happens a lot.

or similar code, so row is invalid for some reason. xwidgets currently disable some redisplay opimizations so it might be an actual emacs bug manifesting without optimizations.

bt 1

* Compute the width of this line. * row->pixelwidth = row->x; for (i = 0; i < row->used[TEXTAREA]; ++i) row->pixelwidth += row->glyphs[TEXTAREA][i].pixelwidth;

(gdb) bt #0 0x000000000045c340 in computelinemetrics (it=0x7fffffff8a20) at xdisp.c:17549 #1 0x00000000004603da in displayline (it=0x7fffffff8a20) at xdisp.c:18792 #2 0x0000000000457646 in trywindow (window=23403045, pos=…, flags=1) at xdisp.c:15399 #3 0x00000000004559c9 in redisplaywindow (window=23403045, justthisonep=0) at xdisp.c:14944 #4 0x0000000000450247 in redisplaywindow0 (window=23403045) at xdisp.c:13152 #5 0x00000000005fdcd9 in internalconditioncase1 (bfun= 0x450208 <redisplaywindow0>, arg=23403045, handlers=12691046, hfun= 0x4501d9 <redisplaywindowerror>) at eval.c:1538 #6 0x00000000004501ba in redisplaywindows (window=23403045) at xdisp.c:13132 #7 0x000000000044f19c in redisplayinternal () at xdisp.c:12706 #8 0x000000000044f9f2 in redisplaypreserveechoarea (fromwhere=7) at xdisp.c:12964 #9 0x0000000000568525 in swallowevents (dodisplay=1) at keyboard.c:4197 #10 0x0000000000422554 in sitfor (timeout=40, reading=1, dodisplay=1) at dispnew.c:5963 #11 0x000000000056512c in readchar (commandflag=1, nmaps=8, maps= 0x7fffffffd3f0, prevevent=12720514, usedmousemenu=0x7fffffffd604, endtime=0x0) at keyboard.c:2689 #12 0x0000000000572c59 in readkeysequence (keybuf=0x7fffffffd850, bufsize= 30, prompt=12720514, dontdowncaselast=0, canreturnswitchframe=1, —Type <return> to continue, or q <return> to quit— fixcurrentbuffer=1) at keyboard.c:9291 #13 0x0000000000562897 in commandloop1 () at keyboard.c:1446 #14 0x00000000005fdb52 in internalconditioncase (bfun= 0x5624b4 <commandloop1>, handlers=12772898, hfun=0x561dab <cmderror>) at eval.c:1493 #15 0x00000000005621ab in commandloop2 (ignore=12720514) at keyboard.c:1157 #16 0x00000000005fd4ce in internalcatch (tag=12768770, func= 0x562185 <commandloop2>, arg=12720514) at eval.c:1247 #17 0x000000000056215e in commandloop () at keyboard.c:1136 #18 0x00000000005618f9 in recursiveedit1 () at keyboard.c:757 #19 0x0000000000561a95 in Frecursiveedit () at keyboard.c:821 #20 0x000000000055fba2 in main (argc=1, argv=0x7fffffffe188) at emacs.c:1704

bt 2

DONE Examine using XComposite rather than GTK off-screen

rendering. This would make xembed widgets work much better. This would probably be rathter difficult, but could open up other interesting possibilities for Emacs. There is an early attempt in xwidget.c, but the X call to redirect to offscreen rendering fails for unknown reasons.

the attempt was further worked on, and the xlib calls replaced with gdk calls, this works better.

In the end I abandoned this aproach. Xwidget-osr is the new aproach.

TODO make the keyboard event code propagation code work.

There is an attempt to provide an api to send keyboard events to an xwidget, but it doesnt currently work very well.

TODO try gtk event creation instead

since that works fine in the webkit osr code. but, oh no, that didn't work for some reason. the widgets seems to receive the event but then the embedded widgets hangs.

http://kegel.com/gtk/button.c

TODO examine some library to synthesize events

xdotool xte xautomation crikey libxdo

TODO webkit raw keyboard event escape

c-c tab could send a raw tab to the webkit instance.

DONE remove the special-case for when the minibuffer is

active. I added some code to reduce the annoying problem display artefacts when making the minibuffer the selected window. This made xwidgets in the buffer go grey or black whenever one did m-x to activate the minibuffer. The coded tried to handle the minibuffer as a special case. That simply wasnt a good idea. Special-casing will never work properly. It is much better to spend time finding solutions that work acceptably in the general case.

DONE disable emacs cursor drawing on top of an active xwidget.

This ought to be rather simple and should improve the visuals a lot.

TODO improve the xwidgets programming interface

so its less of hand-waving affair. This shouldnt be too hard, but I have deliberatley not spent any time on it, since getting the visuals right is much harder. Anyway, I sort of think the interface should be somewhat like it is, except symbols is used instead of integers.

DONE use symbols for xwidget types rather than ints

TODO better lisp based structure for xwidgets

the lisp interface woud be like this:

the data structure would be something like this:

there are some things that arent clear:

stuff that needs to work:

(- do something for all siblings of a xw-view. not atm)

DONE xwidget creation interface

xwidgets are a little bit like emacs processes but also a little bit like emacs images. Therefore its not perfectly obvious how to handle creation. Currently I just use hardcoded identifiers. the real scheme needs to be something else.

Heres a tentative approach:

the problem with this aproach is that its not really legal to reuse xwidget objects by writing several display specs who reference the same xwidget. It could presumably be done but it would just become weird for no real benefit. the big preblem is that the display spec decides the on-screen size, and its not sane to have xwidget views with different sizes. therefore such display specs would need to be catched and dissallowed. Except it is hard because AFAIK the specs don't have an identity as such. A flag in the structure could be set by lookup so the first display attempt would win. but then you can't rewrite the spec to change the size. hmmm. A third approach would be to just allow the 1st spec refering an xw during a redisplay to take effect, the rest are signaled as errors. this wouldnt be too bad.

the other aproach would be to work more like images:

This approach seemed good, but how do I know which instance generates an event if I cant set the id beforehand?

so, therefore, the first two aproach is used.

DONE xwidget creation interface actually

conclusion of above ramblings:

TODO callbacks would be nice

but they need to be handled initially with events for technical reasons. C code can't call Lisp easily. The event handler can call the callback though.

TODO more documentation

There should be user docs, and xwidget contributor docs. The current README is all contributor docs there is now, apart from the code.

CANCELLED look into more ways of displaying xwidgets, like binding them to a

window rather than a point in a buffer. This was suggested by Chidong. This would be a useful addition to Emacs in itself, and would avoid nearly all display issues. I still think the general case is more interesting, but this special case should also be added. The xwidget would then be bound to replace the view of a particular window, and it would only show in that window.

I got the webkit xwidget to work well enough so I dont see the need for this now, except for sockets and I think it can better be dealt with at the lisp level.

DONE MVC mode for xwidgets

It appears unfruitful to chase using the same display mode for all types of xwidgets. Composition is fun but not robust the way I tried to do it.

Instead there should be a set of MVC xwidgets. Each on-screen instance of an MVC widget would be a real GTK widget. The instances would communciate state using signals.

There are drawbacks. There is no inbuilt support for MVC in GTK, so we have to roll our own, which is tedious if not much work for the few cases.

MVC for xembedded application will need support from the applications themselves. Inkscape supports multiple views to the same document, other programs don't. In practice it might not be a big drawback.

DONE figure out what to do with the multiple frames case.

This should be easier to solve with MVC. Surprisingly, this just worked!

DONE how to propagate changes in views to other views?

I used gtk signals, the implementation for sliders works well!

TODO canvas support

heres an interesting comparision of gtk canvases http://live.gnome.org/ProjectRidley/CanvasOverview

ATM there are small hardcoded demos in the code, these should be removed and replaced with working xwgir counterparts.

goocanvas

goocanvas is a gtk canvas implemented using cairo. investigate.

pros:

http://developer.gnome.org/goocanvas/unstable/goocanvas-model-view-canvas.html

export CFLAGS="`pkg-config –cflags goocanvas` -DHAVEGOOCANVAS" export LDFLAGS=`pkg-config –libs goocanvas` ./configure make

I made a hello goo world xwidget so seems doable. I wanted to load a SVG which wasnt immediately straightforward, so I tried clutter. but it turns out the exact same strategy could be used with goocanvas.

clutter

maybe clutter can be used as a canvas? pros:

cons:

(media-explorer in an application that employes the MVC pattern)

http://www.openismus.com/documents/clutter_tutorial/0.9/docs/tutorial/html/sec-stage-widget.html

there is also cool stuff like this: http://gitorious.org/webkit-clutter/webkit-clutter which is an webkit actor for clutter! hmmmmm.

I want to render svg. aparently: librsvg rsvghandlerendercairo(h, cr); ClutterCairoTexture Clutter

export CFLAGS="`pkg-config –cflags clutter-gtk-1.0` -DHAVECLUTTER" export LDFLAGS=`pkg-config –libs clutter-gtk-1.0` ./configure make

compiles but I get: Gtk-ERROR **: GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported

export CFLAGS="`pkg-config –cflags clutter-gtk-0.10` -DHAVECLUTTER" export LDFLAGS=`pkg-config –libs clutter-gtk-0.10` ./configure make

webkit html 5

expose the DOM to lisp or something. The webkit xwidget works pretty well now, so this might be the way ahead.

DONE mvc code crashes after a while

seemingly only when compiling with optimizations. I have no idea why.

Doesn't seem to happen after some code cleanups.

DONE xwidget-resize-at

reimplement so display spec is not involved

DONE display spec validation

it is an error to reuse xwidgets in several buffers or in the same buffer. how do we catch these errors?

so turned out this got solved by using proper lisp objects for xwidgets. yay!

DONE clipping of controllers

Emacs uses a big GTK window and does its own clipping against Emacs windows inside this area. So, in order to layout gtk widgets in emacs windows we must clip thim ourselves.

The following method worked well for a long time:

the only weirdness was that one has to tell gtk the clip area has a window in order to get clipping. This is weird because all gtkwidgets are windows in a sense and a window is almost by definition also a clipping area.

Anyway, in GTK3 the gtkwidgetsethaswindow(GTKWIDGET ( xv->widgetwindow), TRUE); call is ignored.

The gtkeventbox which is documented to have its own window doesnt work either.

http://www.lanedo.com/~carlos/gtk3-doc/chap-drawing-model.html

anyway clipping is rather complicated but seems to finally work okay.

DONE subclass my own clipping widget

http://www.lanedo.com/~carlos/gtk3-doc/GtkWidget.html#gtk-widget-set-has-window mentions that it haswindow can only be called inside a widget implementation.

this wasnt really the issue. allocation was the problem

DONE try scrolled window

clipping does in fact work with gtkscrolledwindowaddwithviewport (xv->widgetwindow, xv->widget); !!

I get unwanted scrollbars in the widget though.

gtkscrolledwindowsetpolicy ( xv->widgetwindow, GTKPOLICYNEVER, GTKPOLICYNEVER);

stops clipping from working!

DONE try viewport

gtkviewport is used in scrolled window so in order to remove scrollbars it should be possible to use viewport directly. however, viewport ignores size requests. or rather the container does.

DONE debug allocation

the container determines how much size to allocate to child widgets.

GtkAllocation galloc; gtkwidgetgetallocation(GTKWIDGET (xv->widgetwindow), &galloc); printf("allocation %d %d , %d %d", galloc.x,galloc.y,galloc.width,galloc.height);

after my clipping attemp shows that my size request is ignored! this might be logical, since the container provided by emacs is a gtkfixed. gtkfixed might choose to heed the widgets size desires and allocate the entire widget size. but we want clipping!

since i cant reasonably expect to change the emacs main container, i can maybe overide the setallocation method in gwfixed, and adjust allocation to clipping if its an xwidget asking for allocation.

  1. DONE subclass gtkfixed

    possibly i need to subclass gtkfixed and override

    void                gtk_widget_size_allocate            (GtkWidget *widget,
                                                         GtkAllocation *allocation);

    http://developer.gnome.org/gobject/stable/howto-gobject.html

    turns out emacs already does this for gtk3 according to jan D: >>For GTK3, Emacs already subclasses GtkFixed, see emacsgtkfixed.[ch].

    • widgets may not be underallocated, aparently

    http://mail.gnome.org/archives/commits-list/2011-April/msg10950.html

    • how to call base class method/chain up

    http://developer.gnome.org/gobject/stable/howto-gobject-chainup.html

    • the allocation modification could happen in the container or the child. it feels more apropiate in the container

    it is however unexpectedy inconvenient to modify allocation because the needed data is private to the base class. to overcome this:

    • run base class method 1st.
    • then, iterate all children, and modify allocation for xwidget children only. x y will then be set.

    JanD pointed out the GTK3 port already has its own subclass, so I modified that one.

DONE clip top

there are four controller edges that potentialy need clipping. I begun with right and bottom edges. clipping them is just a matter of setting the right size of the widgetwindow and also ensure it gets the right allocation from the container.

clipping top (and left) is not equally straightforward. I'm using a viewport now and scroll it the amount that needs to be clipped. however, the viewport is sensitive to changes in allocation, which makes it harder to use the allocation workarounds.

see:

I returned to using a simple gtkfixed for the widgetwindow. with allocation hack and sethaswindow it works. Idea prefer not to have the allocatien hack and it wasnt needed it gtk3 only gtk2. needs further investigation.

various code cleanups

There are many cleanups necessary before any hope of inclusion in Emacs trunk. To begin with, the part of the patch that touches other parts of emacs must be very clean.

DONE use FRAMEGTKWIDGET (f)

rather than gwfixed.

DONE support configure

DONE ifdef all xwidget code

so you can reliably disable the code at compiletime

DONE translate clicks

on onscreen webkit peer to offscreen

maybe http://developer.gnome.org/gdk/stable/gdk-Windows.html#GdkWindow-from-embedder

turned out to be not so hard, captured events, copied them and forwarded them offscreen!

CANCELLED investigate gdkwindowredirecttodrawable

(cancelled this, the current approach seems okay) http://developer.gnome.org/gdk/stable/gdk-Windows.html#gdk-offscreen-window-set-embedder maybe could be used in place of my own copy hacks? to work it must support a chain of redirects, which seems unlikely. the benefit would be that I dont have to spend time optimizing redrawing.

DONE remove xwidgetviews when emacs window is deleted

removing xwidget views when an Emacs window closes is not reliable.

this seems to work

browser xwidget

although embedding a browser is not my primary concern many are interested in this. some suitable browser component needs to be found supporting gtk.

DONE webkit

there is a webkit gtk port. there is no obvious mvc support. http://live.gnome.org/WebKitGtk http://webkitgtk.org/

it might be possible to keep a set of webxits in artificial synchronisation by recursive deep copy of the DOM from one webkit to another. This will be error prone at best though. Another way might be to just use bitmap copy of the "live"instance to the "phantom" instances. the problem of transfering the live view remains though.

export CFLAGS="`pkg-config –cflags webkit-1.0` -DHAVEWEBKIT -g" export LDFLAGS=`pkg-config –libs webkit-1.0` ./configure make

  1. off screen rendering

    export CFLAGS="`pkg-config –cflags webkit-1.0` -DHAVEWEBKITOSR -g" export LDFLAGS=`pkg-config –libs webkit-1.0` ./configure make

    works a little bit but i get errors like:

    (emacs:8362): GLib-GObject-WARNING **: invalid cast from `GdkOffscreenWindow' to `GdkDrawableImplX11'

    set a breakpoint in glog, backtrace seems to indicate webkitViewportAttributesRecompute is the offender.

    maybe try gtk3 variants?

    export CFLAGS="`pkg-config --cflags webkitgtk-3.0 ` -DHAVE_WEBKIT_OSR "
export LDFLAGS=`pkg-config --libs webkitgtk-3.0 `
./configure   --with-x-toolkit=gtk3
make

    crash in gtkwindowgetsize instead. great.

    http://gtkplus-p3.0.sourcearchive.com/documentation/2.91.5-0ubuntu1/testoffscreenwindow_8c-source.html

    after many attempts, the basic issue remains. for some reason the offscreen widget isnt ok when I want to snapshot it, so i simply get emptiness. the surface is only ok sometimes.

    here is a useful debugging snippets:

    // debugging redraw:
//  - the bg colors always change, so theres no error in signal handling
//  - i get this error now and then:
//(emacs:7109): GLib-GObject-WARNING **: invalid cast from `GdkOffscreenWindow' to `GdkDrawableImplX11'
// seems to happen in webkit actually. see README

if(0){ //redraw debug hack. helped a lot in fact. use the with alpha painter below also
  cairo_set_source_rgb(cr, osr_dbg_color, 1.0, 0.2);
  cairo_rectangle(cr, 0,0, xw->width, xw->height);
  cairo_fill(cr);
  osr_dbg_color+=0.1;
  if(osr_dbg_color>1.0)
    osr_dbg_color=0.0;

}

    you need to terminate drawing like this:

    //cairo_set_source_surface (cr, src_pixmap, 0,0); 
//cairo_set_operator (cr, CAIRO_OPERATOR_OVER);

//cairo_paint_with_alpha (cr, 1.0);
//cairo_paint(cr);

    the snippets change background color on oach redraw.

  2. on-screen rendering to separate window

    an alternative might be to open a separate window and snapshot it. the idea is that whatever oddness webkit does so that offscreen rendering doesnt work, doesnt happen on-screen. the window could be opened somewhere not in the way.

CANCELLED firefox

http://www-archive.mozilla.org/unix/gtk-embedding.html seems to be severly bitrotted

heres a newer aproach http://hg.mozilla.org/incubator/embedding/file/29ac0fe51754/gtk/tests/test.cpp

while webkit clearly has the best traction as an embedded, the offscreen rendering issues makes it interesting to see what ff brings to the table.

turned out webkit has as good offscreen support as anyone, see I went with that in the end.

DONE text field support

Emacs captures all keyboard events so text field support isn't super straightforward.

  1. propagate keyboard events

    I have some old hacks for this and they are not good.

  2. use the DOM model

    expose document.activeElement to lisp. This is potentially more interesting than just forwarding keyboard events.

    webkitwebviewgetdomdocument ()

    this is hard it seems. an idea might be to hack elisp support for swig to machine generate the bindings.

  3. DONE inject javascript

    webkitwebviewexecutescript ()

    this works now: (xwidget-webkit-execute-script 5 "document.activeElement.value='test'")

    so it should be possible to do some interesting stuff. execute-script does however not return anything at the interface level so satisfaction is not total:

    http://markmail.org/message/4yowmdgras73z3x5

    maybe https://launchpad.net/gnome-seed

    or this funny hack: <jave> im trying to understand how to interact via javascript to an embedded webkit gtk instance [23:38] <jave> i use webkitwebviewexecutescript() which is nice but doesnt return a value, by design aparently [23:39] <jave> any hints? <lucian> jave: afaik, webkit still doesn't have full gobject bindings [23:48] <lucian> jave: you can hack it up by making the JS modify the title, and read the title from gtk-side <jave> lucian: that was a pretty cool idea!

webkitwebviewloadstring ()

I would like preview of html in a buffer rather than from uri.

DONE simple xwidget-webkit wrapper

so that it could be used for actual browsing :) I dont want to reinvent too many wheels so i'd like to use existing emacs facilities here possible. use bindings similar to w3m(or info)

… and one can of course go on bikeshedding forever. lets keep it simple and extensible, and compatible with other Emacs packages.

the really cool ideas would need Emacs DOM integration, which is not easy.

TODO webkit support webkit signals

  1. DONE particularly document-load-finished

    http://webkitgtk.org/reference/webkitgtk-webkitwebview.html#WebKitWebView-document-load-finished because one might need tell set a title and sizes and things when it loads.

  2. TODO event bug

    Debugger entered–Lisp error: (error "Two bases given in one event")

    hapens sometimes with xwidget events. appears to be when the originating xwidget is offscreen so that the event doesn't get caught by the correct emacs event map.

    maybe I need to set the originating window in the event structure. event.frameorwindow = Qnil; //frame; //how to get the frame here? //TODO i store it in the xwidget now

    since its an offscreen xwidget the buffer local keymap isnt the right place for the handler. some global map should be used.

    onscreen widgets don't have the same issue.

    anyway, seems it'll turn out like this:

    • xwidget-osr stores a callback and user data
    • the event is an implementation detail only and get caught in the topmost event map
    • the event map calls the callback in the xw with the right args.

    we need the event handler at some level because we can't call lisp asynchronously.

  3. TODO navigation signal

  4. TODO new window signal

TODO console messages

http://webkitgtk.org/reference/webkitgtk-webkitwebview.html#WebKitWebView-console-message http://getfirebug.com/wiki/index.php/Console_API#console.count.28.5Btitle.5D.29 because maybe we can make a simple JS REPL that way. (xwidget-webkit-execute-script ( xwidget-webkit-last-session) "console.log('hello')") prints hello to stdout but theres no way to catch stdout from webkit I think other than receiving the signal.

TODO webkit flashkiller by default

while its possible to support plugins in the webkit xwidget, flash has issues on 64 bit, and slows down emacs to a halt with off screen rendering, and of course is not free software. its in the way for real world usage even if its interesting to watch flash animations inside emacs. which should be achieved with Gnash or other free software instead.

http://stackoverflow.com/questions/4885513/prevent-flash-in-cocoa-webview

simply use this api: http://webkitgtk.org/reference/WebKitWebPluginDatabase.html

theres an implementation now but it's not robust enough webkit often crashes taking emacs with it.

TODO webkit downloads

when clicking a download link in Webkit Emacs should take over and handle it from there. Probably need signals. There are Emacs libraries to download things, with wget etc. an url.el facility should be made. "download-requested"

TODO webkit alt-text not handled

XKCD use image-title to display a cartoon comment. These mysteriously don't work ATM. Other mouseovers work though. Maybe webkit tries to open a new window or something, which wont work.

TODO webkit isearch in webkit buffers

have a look at how docview solves it webkitwebviewsearchtext ()

TODO webkit relative references doesn't work

because we handle scrolling in a non-standard way. It does work sort of when theres a html frameset and webkit scrolls by itself.

internal links (page.html#section) do not work see xwidget-webkit-show-named-element

also did some webkit signal work for this.

now it actually works! except for I need to know the Y coordinate of the element to navigate to, and that can either be by "name" or "id" attribute, currently "id" works.

TODO webkit width adjustment handling issue

since there are so many levels of clipping and whatnot in xwidgets sizing issues are difficult.

That should have been enough. but webkit never reports less than what it already has. So currently a webkit view will only growth and not adjust to smaller sizes.

This is not a big problem in practice but is still annoying.

to see the problem surface to http://www.slashdot.org

and then compare by resizing in Epiphany, which is also webkit based.

  1. TODO try putting webkit osr inside a scrolling window

    it seems webkit is supposed to behave differently while embedded in a scrolling window. This is a bit cumbersome because the container stack is already deep.

TODO xwidget webkit allow loading from string from emacs

DONE xwidget-webkit-last-session

was rather hurried. end result is that the lisp layer only really allows for one webkit session.

TODO extract DOM to lisp

then the SHR html renderer from Gnus could render the DOM as created by Webkit.

made a simple oxperimental DOM tree traverser. It can be expanded to return a lisp representation, LDOM.

in order to bring lisp and DOM closer together the LDOM can include a mapping to the originating DOM node. so, find a node in LDOM, and the cell maps to the original DOM. but since the LDOM is a copy it can get out of sync. DOM events might help.

DONE C-X b in other buffer from webkit

bafflingly resets the webkit view to the top. Maybe the window reconfiguration hook code? further mystification is added because it only seems to happen with ido mode enabled.

in comparison with image-mode which does the right thing, I discovered that image-mode has special code to handle scrolling. the browser mode and image mode has some similarities.

I made some delegation code frrom webkit mode to image mode.

TODO url-browse improvement

sindikat: site.com and http://site.com should be equivalent (simple site.com throws error)

Yes, but its unclear at what level in Emacs to do this properly. I added a url-tidy function as a start.

this should be further improved:

TODO sindicat notes

Here are some comments from user "sindikat" and my replies

The problem is the size communication between Emacs and Webkit.

True, I hadn't noticed. Thanks.

Size communication.

Need to verify with some other webkit browser.

This is because I inherit from Image view mode. I kind of like it so we can add an option for it.

The cause is the lack of return value in the webkit evaluation API. Ive made some fixes.

Indeed, this would require better DOM integration.

Probably because the Image mode derivative is mostly a hack. fixed now I think.

Either use the webkit search but that doesn't feel right. It would be better to expose the DOM and search that.

Keyboard integration is the unloved step-child of xwidgets, unfortunately.

TODO xwidget image display spec compatibility

some history: the first version of the xwidget display spec was the same as an image spec. This turned out not to be fantastic because an xwidget is both like a process and like an image. it has a separate existence from display. So now the xwidget display spec is just a pointer to a xwidget. But then some useful functionality in Emacs can't be reused for xwidget, in particular image-mode.

Maybe a new image type could be added that was a wraper on an xwidget. Then image mode could be reused for webkit mode.

I tried some adaptor code in xwidget.el so webkit mode now delegates to image mode but its a kludge.

TODO some flickering during redisplay of sockets

with gtk3 an size allocation workaround is used. this seems maybe to result in flickering sizewize y-axis with the xwidget socket type. The webkit xwidget doesn't seem similarily afflicted.

the size allocation workaround works by 1st running the ordinary allocation then modifying the results. its done this way to minimize the copy paste index from the base class. it might be that the original allocation has a brief time window to show itself.

tried to modify the allocation hack so it doesn't call allocate twice. this doesn't seem to help flicker at all aparently so the hypothesis falls. Maybe then a socket simply doesn't like being clipped by gtkfixed.

TODO xwidget view reaping too agressive

hide an emacs window for a while and return to it. the xwidget might get reaped and a new socket thus created.

DONE try out OSR for sockets

didn't work too well in the inkscape case. it might be that some other bitmap copy method works better though.

basically sockets doesn't like to be offscreen because they want their own gdk window.

DONE synchronise emacs background with xwidget color

fine-tuning to reduce flicker.

isn't needed if emacs bg erase disabled

DONE xwidgets doesn't work during bootstrap all of a sudden

might be some annoying local issues with my install because it is not reliably reproducible. (went away during merges)

CANCELLED low impact xwidget based image viewer

(cancelled this because it no longer seems like a good idea) for instance to render SVG using webkit, or some other canvas. that way it would be possible to merge to trunk in stages.

so, webkit could be used to display the SVG. the display spec for images would be used. multiple webkits would be used rather than offscreen rendering, so it would be GTK2 compatible.

DONE xwidget movement doesn't work all of a sudden

this used to work great. now it doesn't.

suspects:

xdrawxwidgetglyphstring probably because of scroll optimization. movement detection could possibly be moved to producexwidgetglyph(not)

no longer helps: (setq inhibit-try-window-id t) (setq inhibit-try-window-reusing t)

workaround: (run-with-timer 1 1 'redraw-display)

seems to work: inhibiting scrollingwindow(). and this seem to be enough to restore the old behaviour, GLYPHDEBUG doesn't seem needed.

DONE GLYPHDEBUG doesn't work

was stupid accidental line removal that was hard to spot

TODO osc xwidget example

a couple of xwidget sliders that control a csound/supercollider song with osc. so, for that to work we need slider callbacks to work. when a slider changes send an osc message. use ocssend:

oscsend localhost 7777 /sample/address iTfs 1 3.14 hello

or better: http://delysid.org/emacs/osc.el

sliders could be defined in csound comments or something to illustrate the point. or if real fanciness is desired parse the csound source with Semantic and provide a control buffer corresponding to the defined controls.

Added: [2011-08-11 Thu 10:53]

DONE SEB

the SEB site does something funny so I can't insert text in fields. aparently document.activeElement.value doesn't work with framesets.

seems to work using the ugly javascript in xwidget-webkit-activeelement-js

support downstreams

http://aur.archlinux.org/packages.php?ID=53902 http://gpo.zugaina.org/app-editors/emacs-xwidget/ChangeLog

DONE the proof of concept canvas code should be disabled by default.

TODO advi

active dvi viewer. investigate if it could be xwidgetified. advi supports embedding inside presentations.

cairo configuration support

gtk3 brings in cairo on Fedora, but apparently not on all plattforms. pkg-config –cflags cairo

TODO splint

splint -Demacs -DHAVECONFIGH -I. -I/home/joakim/buildmyprojs/emacsnew/emacs.bzr/xwidget.mint/src -I../lib -I/home/joakim/buildmyprojs/emacsnew/emacs.bzr/xwidget.mint/src/../lib -DGSEALENABLE -I/usr/include/gtk-3.0 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/include/freetype2 -I/usr/include/alsa -I/usr/include/librsvg-2.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/cairo -I/usr/include/libpng12 -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/ImageMagick -I/usr/include/libxml2 -I/usr/include/dbus-1.0 -I/usr/lib64/dbus-1.0/include -DGSEALENABLE -I/usr/include/webkit-3.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/gtk-3.0 -I/usr/include/libsoup-2.4 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/include/libxml2 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -DORBIT2=1 -I/usr/include/gconf/2 -I/usr/include/orbit-2.0 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/freetype2 xwidget.c

TODO 32 bit bug

user reports that xwidgets segfaults on the 32 bit Mint distribution but not the 64 bit. Mint is an Ubuntu derivative. I got some VirtualBox images to test with.

DONE youtube

http://www.youtube.com/watch?v=DZdUgjEx_dQ&html5=1 html5 makes it work without stupid flash plugins!

TODO clicking on an webkit xwidgets

doesn't make the window active. this leads to problems.

DONE "g" should default to current url

"g" runs xwidget-webkit-browse-url which gets its interactive argument from browse-url-interactive-arg. this might need a new optional argument.

http://test

TODO anything/helm support

hook so anything/helm can filter browser history.

TODO new relative url code sometimes fail

http://www.dilbert.com

TODO input field enhancements

password field.

was straightforward

textarea

less straightforward. I would like it to work like emacs-w3m, where a new editing buffer is opened. on c-c, the buffer is closed and the browser field updated. however, it's not immediately obvious how to store the reference to the field reliably.

furthermore the current code doesn't seem to propagate linefeed properly to text areas.

DONE bug in current navigation handler

on www.dn.se Debugger entered–Lisp error: (args-out-of-range "http://platform.twitter.com/widgets/hub.html" 54 357) match-string(1 "http://platform.twitter.com/widgets/hub.html") xwidget-webkit-callback(48890368 navigation-policy-decision-requested) xwidget-event-handler() call-interactively(xwidget-event-handler nil nil)

TODO how to set the name of a webkit buffer?

not obvious because, the buffer isn't created immediately and there is a callback that sets the buffer name automatically

TODO how to find next field in tab order?

TODO unique buffer names

the webkit xwidgets renames the buffer after load but not uniquely so it sometimes fails.

TODO kill the offscreen webkit xwidgets when last view killed

The offscreen xwidgets is currently kept around even if the xwidgets views are all gone. this is a general problem and it requires actions on the behalf of the application to resolve.

In the case of webkit it is currently possible to get errors like these:

Debugger entered–Lisp error: (error "Selecting deleted buffer") xwidget-webkit-callback(60925380 navigation-policy-decision-requested) xwidget-event-handler() call-interactively(xwidget-event-handler nil nil)

because the last view is gone and the offscreen widgets is still generating events.

In the case of webkit it is okay to kill the offscreen widgets completely when the user kills the last view window because it would be unexpected by the user to see it pop up again. This is not true in the general case.

DONE xwidgets debugging log

currently theres a lot of debugging traces using "message" which is annoying. Instead put them in a separate trace buffer. (see xwidgetbuffer)

TODO make garbage collect work for xwidgets

when an xwidget is removed from xwidget-alist, and there are no other references(mostly views) the xwidget should be garbage collected.

special finalization would go into gcsweep()

TODO embedding evince

http://developer.gnome.org/libevview/3.2/libevview-ev-view.html would be useful for reading PDF:s and other document types. it would work the same way webkit embedding works.

TODO support gobject introspection

https://live.gnome.org/GObjectIntrospection/ supporting gobject introspection would mean that more gtk widgets could be tried out with less effort, and also that the build process and runtime support would be easier. The drawbacks are small: somewhat slower execution, and difficulty in providing elisp bindings for introspection.

https://live.gnome.org/GObjectIntrospection/HowToWriteALanguageBinding

http://developer.gnome.org/gi/unstable/gi-girepository.html http://developer.gnome.org/gi/unstable/gi-overview.html

In order for GIR to work, it needs the namespace and class of a widget. This is used to access the typelib file, which contains the introspection data. The namespace and class is stored as a property on the lisp symbol handle used by xwidgets to identify the widget class.

This snippet sets the needed :xwgir-class property, and calls the setzoomlevel method:

M-x xwidget-webkit-browse-url RET www.emacswiki.org RET

Then eval the following:

;;load the webkit typelib (xwgir-require-namespace "WebKit" "2.0")

;;provide the metadata needed so xwgir can work with the webkit-osr xwidget (put 'webkit-osr :xwgir-class '("WebKit" "WebView")) ;;call the method (xwgir-call-method (xwidget-at 1) "setzoomlevel" '(3.0))

It's also possible to create widgets dynamically, by using introspection to call a widget constructor(from xwidget-test.el):

(defun xwgir-test () (interactive) (xwgir-require-namespace "Gtk" "3.0") (put 'color-selection :xwgir-class '("Gtk" "ColorSelection"))

(xwgir-demo-a-xwgir-button) (xwgir-call-method (xwidget-at 1) "setlabel" '( "xwgir set label!")) )

Current limitation:

TODO xwgir create components with more advanced constructor

so this opens up an entire new can of beans.

explain by example: lets say we want to create agtkhscale on screen. its a slider. https://developer.gnome.org/gtk3/stable/GtkHScale.html we can already create buttons, so sliders shouldnt be much more advanced right? wrong.

the simplest slider constructor looks like: GtkWidget * gtkhscalenew (GtkAdjustment *adjustment);

so in order to call it, we must be able to forward arguments to the constructor. this is almost already done, but we lack the ability to pass object instances, only simple types atm.

we need to be able to create GtkAdjustment https://developer.gnome.org/gtk3/stable/GtkAdjustment.html

this we can already almost do, because an xwidget is a gir object with some decorations. we also store the decorated gir object in an array, retrievable from lisp.

In order for this to be usable in practice, we need some changes:

DONE investigate gdkoffscreenwindowsetembedder()

https://developer.gnome.org/gdk/unstable/gdk-Windows.html

,—-

Offscreen windows are more general than composited windows, since they
allow not only to modify the rendering of the child window onto its
parent, but also to apply coordinate transformations.
To integrate an offscreen window into a window hierarchy, one has to
call gdkoffscreenwindowsetembedder() and handle a number of
signals. The "pick-embedded-child" signal on the embedder window is
used to select an offscreen child at given coordinates, and the
"to-embedder" and "from-embedder" signals on the offscreen window are
used to translate coordinates between the embedder and the offscreen
window.
For rendering an offscreen window onto its embedder, the contents of
the offscreen window are available as a pixmap, via
gdkoffscreenwindowgetpixmap().

`—-

okay, [2013-04-03 Wed] I finally suceeded in this approach! it was pretty hard to make it work and currently works like this:

event forwarding is done automatically!

BUT its not really super, because it only works well with a single embedder.

perhaps the strategy could be refined:

[2013-04-04 Thu] I had a strategy working for a xwgir button but not a webkit. setembedded in the motion event handler for the xv. it even works for 2 frames! but not webkit :(

[2013-04-05 Fri] it works for xwgir osr components, but not for webkit. Webkit retains the previous event forwarding system.

Now it works like this:

The above approach has been tested for xwgir created buttons and seems to work. it doesnt work for webkit, so the old scheme is preserved for webkit.

DONE investigate git-remote-bzr

TODO Eli Zaretski Xwidget review

These are some notes from a review of the xwidget code from Eli Zaretski. See emacs-devel for the full thread. HEre I have just converted Elis notes to TODO items.

From: joakim@verona.se To: Eli Zaretskii <eliz@gnu.org> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org Subject: Re: [Emacs-diffs] xwidget updated (1d8b8a2 -> 5f46725) References: <20141226164113.11620.38682@vcs.savannah.gnu.org> <jwv1tnli0dw.fsf-monnier+emacsdiffs@gnu.org> <m3h9wht7oj.fsf@exodia.verona.se> <831tnjlpts.fsf@gnu.org> X-Draft-From: ("nnimap+naru:imapmail.lists.emacs-devel" 93634) Date: Mon, 29 Dec 2014 10:48:00 +0100 In-Reply-To: <831tnjlpts.fsf@gnu.org> (Eli Zaretskii's message of "Sun, 28 Dec 2014 18:08:31 +0200") Message-ID: <m3y4pqss6n.fsf@exodia.verona.se> User-Agent: Gnus/5.130012 (Ma Gnus v0.12) Emacs/24.4.50 (gnu/linux) –text follows this line– Eli Zaretskii <eliz@gnu.org> writes:

>> From: joakim@verona.se >> Date: Sat, 27 Dec 2014 16:48:44 +0100 >> Cc: emacs-devel@gnu.org >> >> Stefan Monnier <address@hidden> writes: >> >> > Hi Joakim, >> > >> > BTW, what's the status of this branch (w.r.t to its mergeability into master)? >> >> Short answer: I'm trying to figure that out now. >> >> Longer answer: >> >> I think its pretty okay. Theres a problem with automatic >> resizing of webkit that sometimes get so big emacs crashe I would like >> to fix though. >> >> I'm not sure how to do the actual merge. I think cherry picking or >> something would be better than a plain merge. >> >> Also, since I'm pretty blind to the flaws the code has by now, it would >> be nice with some maintainer criticism. > > Thank you for your work. Please find a few comments and questions > below, all based solely on reading the source.

I'm extatic that you found time to review it! Thanks Eli!

Giving a thorough reply will take time, so I just include replies I could give quickly below.

>

TODO 1) In dispextern.h:'struct it' you made this addition to the iterator

> structure: > > @@ -500,6 +504,9 @@ struct glyph > * Image ID for image glyphs (type == IMAGEGLYPH). * > int imgid; > > +#ifdef HAVEXWIDGETS > + struct xwidget* xwidget; > +#endif > * Sub-structure for type == STRETCHGLYPH. * > struct > { > > This might be a problem, because several places in the display > engine make local copies of the 'struct it' object, which will > duplicate the pointer you added, so you will have 2 or more pointers > to the same object. If one of the copies of the pointer is used to > modify the 'struct xwidget' object, or free it, the other copies > will be affected, which the code doesn't expect. Note that images, > for example, store only their numerical ID in the iterator > structure, not a direct pointer to an image. >

DONE 1b. Also, you added a similar pointer to the iterator stack entry:

> > @@ -2379,6 +2396,13 @@ struct it > struct { > LispObject object; > } stretch; > +#ifdef HAVEXWIDGETS > + * method == GETFROMXWIDGET * > + struct { > + LispObject object; > + struct xwidget* xwidget; > + } xwidget; > +#endif > } u; > > But that pointer seems to be unused, so I guess it should be > deleted. >

removed the unused pointer

TODO 2) In dispnew.c:updatewindow you added a call to

> xwidgetendredisplay. I think this call should be made before we > call updatewindowendhook, because when that call is made, the > redisplay interface implementation assumes the window is already up > to date, whereas xwidgetendredisplay still manipulates portions > of the display (AFAIU). > – I put the call at the end of the function originally because xwidgetendredisplay is supposed to be safe. It only tries to discover what happened to the glyph matrix, and changes the xwidget state acccordingly. xwidgetendredisplay wont in itself modify the emacs structures. So while it might work to move the call, it should be safe where it is, and moving it will require more testing.

DONE 3) A few places (for example, xdisp.c:handlesingledisplayspec)

> process xwidget display elements even on non-GUI frames – does > that mean xwidget.c will be compiled even in –without-x > configurations of Emacs? If not, you need to condition that code > on HAVEWINDOWSYSTEM, like we do with images, for example.

No the code shouldn't be compiled if we dont HAVEWINDOWSYSTEM. Thanks for the catch!

–

I attempted to improve this in 4105b6a Improved configury. The configure script now checks for gtk3 and window system. dont enable xwidget unless we have these.

TODO 4) xdisp.c:producexwidgetglyph seems to need some cleanup: it's

> basically a copy of produceimageglyph, and at least some of the > code there is not needed with xwidgets, I think. > OTOH, if indeed xwidgets are very similar to images, perhaps we > should have only one method that handles both. – so far it has been easier and safer to manage a separate code path. I agree that refactoring these methods wouold be a good thing eventually.

DONE 5) xdisp.c:producexwidgetglyph needs to account for bidirectional

> display in the same way produceimageglyph does: swao the left and > right box edges, and populate the bidi members of struct glyph.

I havent thought about bidi at all. Do you have a simple test case? — I tried copying some arabic from the hello file into the (xwidget-demo-a-button) buffer. Text went all sorts of funky places and i couldnt make much sense of it.

fixed, 5769602 bidi bugfix

DONE 6) Did you test what happens with xwidgets when the lines are

> truncated, and only part of the xwidget fits on the line? Are the > results reasonable? I see that producexwidgetglyph does attempt > to crop the xwidget to fit in the line, but then displayline > should handle xwidget glyphs the same as it does with image glyphs, > when it decides how to go about truncation/continuation.

Truncation works same as for an image, which I think is reasonable. Or did you mean something else?

A quick way to test it:

m-x xwidget-browse-url RET http://www.fsf.org RET

before the xwidget. the xwidget will scroll right, and get truncated when it hits the frame border.

I dont understand your comment about displayline. — I tried continuation character, by inserting a button, and scrolling it right. it seemed to work.

TODO 7) xwidget.c:make-xwidget seems to support xwidgets only in a buffer.

> What about strings? If strings aren't going to be supported, then > the 'object' member of the iterator stack entry for xwidgets is not > needed.

Hmm okay. — but i can kill and yank a string with a xw in it? doesnt that mean strings are supported?

DONE 8) Do we really need to expose xwgir-require-namespace? Can't

> something like that be done automatically under the hood?

The xwgir stuff is eventually supposed to work like an FFI. So something like it will be needed.

TODO 9) xwgir-xwidget-call-method needs the method as a string. Wouldn't

> it be better to use a symbol here? Strings are more expensive to > compare, e.g. if some code needs to manage methods.

Okay.

DONE 10) Several places in xwidget.c use Lisp string data without first

> verifying it's a string. Examples include > xwidget-webkit-execute-script and xwidget-webkit-goto-uri.

Yes, I'm lazy, sorry. – CHECKSTRING(script); >

DONE 11) The doc strings of functions exposed to Lisp that are defined on

> xwidget.c are not yet finished.

Yes.

DONE 12) A question about configuration: are xwidgets only supported in a

> GTK3 compiled Emacs, or also in other builds?

xwidgets were originally developed on GTK2, then ported to GTK3. The code only works on GTK3 now, so theres lots of potential cleanup.

AFAICS theres no real obstacle for getting xwidgets to work on whatever windowing system. Off screen rendering, and some other things would be needed, but I think most modern toolkits support that.

OTOH, I think it would perhaps be easier to just use GTK3 on the target build.

Eli, how difficult do you suppose it would be to get a GTK3 emacs running on Windows? — see 4105b6a Improved configury.

TODO 13) A minor stylistic nit: the code style is somewhat different from

> the GNU Coding Standard: no space between the function name and > the left parentheses that follows, opening brace of a block at > the end of a line rather than on the next line, comments that > don't end with a period, etc.

Okay.

TODO 14) Finally, there are a lot of places in the code with FIXME's,

> TODO's, fragments that are commented out, debugging printf's, and > other left-overs that I suggest to clean up before the merge. >

Yes.

> Thanks again for working on this.

And thanks again for the review!

–