amette__: joined #tikiwiki
left #tikiwiki
Tiki-KGB: 03lindonb r62297 04trunk/templates/access/include_ticket.tpl
[KIL] Delete include_ticket.tpl file as it's function has been replaced by the smarty ticket function. Redo of r62266 now that all uses have apparently been deleted.
03lindonb r62298 10trunk/tiki-setup_base.php * Remove assign of smarty variable no longer needed
fabricius: joined #tikiwiki
Telesight: joined #tikiwiki
jonnyb: joined #tikiwiki
nelsonko: joined #tikiwiki
left #tikiwiki
jonnyb: polom
rescue script about half done, going to commit it as it is so far (into trunk)
Tiki-KGB: 03jonnybradley r62299 03trunk/doc/devtools/rescue_wiki_changes_from_elastic.php
Part one of a script to rescue wiki page updates from an elastic search index - for data recovery for doc.tiki.org
Here we get all the activity stream wiki page events over a period of time (novemeber to april) - thanks to amette, luciash and others
TODO: The actual page updates next
jonnyb: so amette_ - are you about? i have a script ready for testing (nearly)...
oops, it just broke ;)
amette__: joined #tikiwiki
plm
hi jonnyb - let me check out the code and check out what you did
jonnyb: hi amette__
it's sort of wokring but i haven't committed the actual page creation part yet
just trying to get the version numbers working properly
amette__: ok, how does it work? Does it use the configured mysql and elastic?
jonnyb: it uses the usual db defined in local, but gets the info from an elastic url you can specify (defaults to localhost:9200 though)
do 'php doc/devtools/rescue_wiki_changes_from_elastic.php help rescue' for the help
amette__: alright, sounds cool. I'll check it out on the doc server then (as the database is there) and connect to the remote elastic as a cli parameter then.
jonnyb: i thought i'd see if i could make a devtools script using the funky symfony console application stuf, so we get coloured output and progress pabrs etc - seems to work! :)
amette__: Getting a professional tool out of this - nice one! :)
jonnyb: well, i thought it would make it more fun! :)
amette__: ;)
ok.... maybe I should actually do this on a completely different machine... and then later just dump, scp and replay the resulting database... *thinkingoutloud*
jonnyb: yes, i think would be wise
-: amette__ agrees
jonnyb: i'm getting some indexing errors but i have es5 installed here (for extra pain)
amette__: right, had similar issues with feature regressions in elastic - decided to downgrade it then
jonnyb: amette__: note that the version committed to trunk so far doesn't actually create or update any pages, so you should be fairly safe... :)
amette__: alright, cool! :)
jonnyb: odd that some of the edits don't seem to have old version info indexed - but maybe that was added later (i think i might have added it for 16.x) - so for the missing ones i'll just guedd (version-1)
amette__: sounds reasonable... and good solution... or should one do (version-2) so it can be distinguished? or does that sound really dirty and consistency killing?
jonnyb: yes, that's what i'm currently working out - i seem to get the current version duplicated...
ok, here comes the dangerous bit! :D
Tiki-KGB: 03jonnybradley r62300 10trunk/doc/devtools/rescue_wiki_changes_from_elastic.php
Part two of a script to rescue wiki page updates from an elastic search index - for data recovery for doc.tiki.org
Here we update (or create) the wiki pages - only lightly tested so far, not for use on "live" data yet :)
jonnyb: there you go amette__ (feeling brave?)
amette__: hehe, sure... still setting up the test environment... hang on a bit
jonnyb: if you get jip from the indexing turning off incremental indexing should help (and run quicker, actually)
making t now... brb
amette__: alright, enjoy!
jonnyb: i'm getting errors from aliasses being duplicated in structures, but don't seem to be able to do anything about it...
-: amette__ is fighting with getting MySQL into shape
fabricius: joined #tikiwiki
amette__: ok, I think I'm getting there... currently importing the DB dump from November and so far no errors.
jonnyb: joined #tikiwiki
amette__: jonnyb : there's no --dry-run or such? Something that would tell me what will happen, but not do anything? That would be handy, cause replaying the database takes ages.
-: amette__ should probably just revert to the previous svn revision
jonnyb: hmmm
ok, let me add a dry-run option, shoudln't take long
amette__: cool, thanks
jonnyb: actually i'll add a confirm option and do dry run by default
amette__: superb!
jonnyb: ok, there you go
amette__: w00t, nice one, bro! :)
Tiki-KGB: 03jonnybradley r62301 10trunk/doc/devtools/rescue_wiki_changes_from_elastic.php
Part three of a script to rescue wiki page updates from an elastic search index
Add confirm option and use dry run mode by default, no changes to pages saved without the -c option (thanks scaredy-cat amette ;)
jonnyb: ah, there it is amette__
i was guessding jusdt an edit or create summary is ok, youi don't want a full diff i hope ;)
amette__: yep, very nice output, well done! (if you wanna be ultra-diff-fancy, add it in with -v ;) )
I have multiple times:
Page: “PhantomJS and CasperJS” (version 11) “Page created by rescue script 2017-04-22T17:07:34+00:00”
just with different version (12, 13, 14...) - shouldn't those be edits instead of creations then?
Tiki-KGB: 03jonnybradley r62302 10trunk/doc/devtools/rescue_wiki_changes_from_elastic.php
Script to rescue wiki page updates from an elastic search index
Make the dry run output less noisy
jonnyb: amette__: no, the pages don't get created in dry run mode
amette__: aah, so it doesn't realise that the second action is a modification intead of a creation!?
jonnyb: if the page exists it updates it, if not it creates it (so i could test it on an empty database)
fabricius: joined #tikiwiki
amette__: Great work, jonnyb! This starts looking really good! :)
jonnyb: think i'm done for now, need food, mbbl or tomorrow...
fabricius: Hey guys, I have seen a rising number of doc.tiki.org edits ... whats happening? A community wide documentation saving session?
“Page created by rescue script 2017-04-22T17:07:34+00:00”
This is more than awsome.
You are so cool guys. THX!
amette__: fabricius : that's a test run on nextdoc. Did you get notification e-mails from that?
Dekthro: joined #tikiwiki
fabricius: amette__: not sure .. what I postest was a citation from the IRC above ... I got this type of notivicatons: "wiki page “Wiki Plugin de” changed by luciash d' being wiki page noreply@tiki.org"
amette_ amete__ ^
amette__: I was referring to: "[20:14:39] fabricius: Hey guys, I have seen a rising number of doc.tiki.org edits"
fabricius: I did *NOT* get a notification like that: “Page created by rescue script 2017-04-22T17:07:34+00:00”
amette__: yes, that's from the script
but where did you get the rising number from?
fabricius: This I did read at 19:12:56 - amette__: I have multiple times ...
amette__: aha
thanks for the confusion
fabricius: sry, did not want to confuse you
:)
amette__: well, we got it cleared up
fabricius: joined #tikiwiki
joined #tikiwiki