amette__: joined #tikiwiki
left #tikiwiki Tiki-KGB: 03lindonb r62297 04trunk/templates/access/include_ticket.tpl
[KIL] Delete include_ticket.tpl file as it's function has been replaced by the smarty ticket function. Redo of r62266 now that all uses have apparently been deleted.
03lindonb r62298 10trunk/tiki-setup_base.php * Remove assign of smarty variable no longer needed fabricius: joined #tikiwiki Telesight: joined #tikiwiki jonnyb: joined #tikiwiki nelsonko: joined #tikiwiki
left #tikiwiki jonnyb: polom
rescue script about half done, going to commit it as it is so far (into trunk) Tiki-KGB: 03jonnybradley r62299 03trunk/doc/devtools/rescue_wiki_changes_from_elastic.php
Part one of a script to rescue wiki page updates from an elastic search index - for data recovery for doc.tiki.org
Here we get all the activity stream wiki page events over a period of time (novemeber to april) - thanks to amette, luciash and others
TODO: The actual page updates next jonnyb: so amette_ - are you about? i have a script ready for testing (nearly)...
oops, it just broke ;) amette__: joined #tikiwiki
plm
hi jonnyb - let me check out the code and check out what you did jonnyb: hi amette__
it's sort of wokring but i haven't committed the actual page creation part yet
just trying to get the version numbers working properly amette__: ok, how does it work? Does it use the configured mysql and elastic? jonnyb: it uses the usual db defined in local, but gets the info from an elastic url you can specify (defaults to localhost:9200 though)
do 'php doc/devtools/rescue_wiki_changes_from_elastic.php help rescue' for the help amette__: alright, sounds cool. I'll check it out on the doc server then (as the database is there) and connect to the remote elastic as a cli parameter then. jonnyb: i thought i'd see if i could make a devtools script using the funky symfony console application stuf, so we get coloured output and progress pabrs etc - seems to work! :) amette__: Getting a professional tool out of this - nice one! :) jonnyb: well, i thought it would make it more fun! :) amette__: ;)
ok.... maybe I should actually do this on a completely different machine... and then later just dump, scp and replay the resulting database... *thinkingoutloud* jonnyb: yes, i think would be wise -: amette__ agrees jonnyb: i'm getting some indexing errors but i have es5 installed here (for extra pain) amette__: right, had similar issues with feature regressions in elastic - decided to downgrade it then jonnyb: amette__: note that the version committed to trunk so far doesn't actually create or update any pages, so you should be fairly safe... :) amette__: alright, cool! :) jonnyb: odd that some of the edits don't seem to have old version info indexed - but maybe that was added later (i think i might have added it for 16.x) - so for the missing ones i'll just guedd (version-1) amette__: sounds reasonable... and good solution... or should one do (version-2) so it can be distinguished? or does that sound really dirty and consistency killing? jonnyb: yes, that's what i'm currently working out - i seem to get the current version duplicated...
ok, here comes the dangerous bit! :D Tiki-KGB: 03jonnybradley r62300 10trunk/doc/devtools/rescue_wiki_changes_from_elastic.php
Part two of a script to rescue wiki page updates from an elastic search index - for data recovery for doc.tiki.org
Here we update (or create) the wiki pages - only lightly tested so far, not for use on "live" data yet :) jonnyb: there you go amette__ (feeling brave?) amette__: hehe, sure... still setting up the test environment... hang on a bit jonnyb: if you get jip from the indexing turning off incremental indexing should help (and run quicker, actually)
making t now... brb amette__: alright, enjoy! jonnyb: i'm getting errors from aliasses being duplicated in structures, but don't seem to be able to do anything about it... -: amette__ is fighting with getting MySQL into shape fabricius: joined #tikiwiki amette__: ok, I think I'm getting there... currently importing the DB dump from November and so far no errors. jonnyb: joined #tikiwiki amette__: jonnyb : there's no --dry-run or such? Something that would tell me what will happen, but not do anything? That would be handy, cause replaying the database takes ages. -: amette__ should probably just revert to the previous svn revision jonnyb: hmmm
ok, let me add a dry-run option, shoudln't take long amette__: cool, thanks jonnyb: actually i'll add a confirm option and do dry run by default amette__: superb! jonnyb: ok, there you go amette__: w00t, nice one, bro! :) Tiki-KGB: 03jonnybradley r62301 10trunk/doc/devtools/rescue_wiki_changes_from_elastic.php
Part three of a script to rescue wiki page updates from an elastic search index
Add confirm option and use dry run mode by default, no changes to pages saved without the -c option (thanks scaredy-cat amette ;) jonnyb: ah, there it is amette__
i was guessding jusdt an edit or create summary is ok, youi don't want a full diff i hope ;) amette__: yep, very nice output, well done! (if you wanna be ultra-diff-fancy, add it in with -v ;) )
I have multiple times:
Page: “PhantomJS and CasperJS” (version 11) “Page created by rescue script 2017-04-22T17:07:34+00:00”
just with different version (12, 13, 14...) - shouldn't those be edits instead of creations then? Tiki-KGB: 03jonnybradley r62302 10trunk/doc/devtools/rescue_wiki_changes_from_elastic.php
Script to rescue wiki page updates from an elastic search index
Make the dry run output less noisy jonnyb: amette__: no, the pages don't get created in dry run mode amette__: aah, so it doesn't realise that the second action is a modification intead of a creation!? jonnyb: if the page exists it updates it, if not it creates it (so i could test it on an empty database) fabricius: joined #tikiwiki amette__: Great work, jonnyb! This starts looking really good! :) jonnyb: think i'm done for now, need food, mbbl or tomorrow... fabricius: Hey guys, I have seen a rising number of doc.tiki.org edits ... whats happening? A community wide documentation saving session?
“Page created by rescue script 2017-04-22T17:07:34+00:00”
This is more than awsome.
You are so cool guys. THX! amette__: fabricius : that's a test run on nextdoc. Did you get notification e-mails from that? Dekthro: joined #tikiwiki fabricius: amette__: not sure .. what I postest was a citation from the IRC above ... I got this type of notivicatons: "wiki page “Wiki Plugin de” changed by luciash d' being wiki page noreply@tiki.org"
amette_ amete__ ^ amette__: I was referring to: ‎"[20:14:39] ‎fabricius‎: Hey guys, I have seen a rising number of doc.tiki.org edits" fabricius: I did *NOT* get a notification like that: “Page created by rescue script 2017-04-22T17:07:34+00:00” amette__: yes, that's from the script
but where did you get the rising number from? fabricius: This I did read at 19:12:56 - amette__: I have multiple times ... amette__: aha
thanks for the confusion fabricius: sry, did not want to confuse you
:) amette__: well, we got it cleared up fabricius: joined #tikiwiki
joined #tikiwiki