[Tool] FTL Error Checker

[Vhati]

on my Mac, EC has 100% usage of one CPU core and occupies about 600MB of RAM while being idle

Whoa. The RAM usage could be explained by the re-created XML structure that the checker uses internally to detect errors.

C'mon, 600 megs!? There is no way text is being handled that inefficiently.

But the CPU usage has me puzzled...

Pegging the CPU usually means a thread is doing an endless loop without blocking/sleeping.

Incidentally, I tracked down such a bug yesterday in the WIP Overdrive engine, the render thread couldn't paint the screen fast enough to meet the FPS it wanted, so it never had an opportunity to rest. I lowered the desired frame rate a little and CPU usage dropped from 100% to 0%.

[ApexMods]

for some reason it wrongly reports false description lengths for bomb weapons with 17-part image strips ("Mismatched frame count declared in anim and frame count in anim sheet (17 != 9)")

The checker compares the number of frames declared by length= attribute in <desc> child in <anim> tags:

Code: Select all

<desc length="#" x="#" y="#"/>

and the frame count acquired by dividing width (w) by frame width (fw) of the linked <animSheet>:

Code: Select all

<animSheet name="" w="#" h="#" fw="#" fh="#">...</animSheet>

I'm not exactly positive that this is how it works, and it seems to have no effects in-game... But most anims/animsheets follow this rule *shrugs*

Yes, I'm aware it should work through calculating the description length by dividing the image, and in all other cases it seems to do that just fine, but for some odd reason it won't divide the 17-part bomb animations properly. Have a look:

Code: Select all

<animSheet name="foo" w="595" h="47" fw="35" fh="47">foo_strip17.png</animSheet>
<weaponAnim name="foo">
   <sheet>foo</sheet>
   <desc length="17" x="0" y="0"/>
   [...]
</weaponAnim>


This is an example where EC will throw the aforementioned error, although 595 pixels divided by 35 pixels is... 17, and not 9.

[Vhati]

the stripping of all multiple line breaks (and spaces ?) that turns any formatted paragraphs of code and comments(!) into one huge wall of text

I'm trying to narrow down where the problem is, the way SMM generates text, or the way the Error Checker is showing it.

Could you compare EC's view with the way SMM's File-"XML Sandbox" displays, say events.xml?

I never had any issues with UTF-8

UTF-8 closely resembles Windows-1252 for certain common characters. This leads to the illusion that UTF-8 works. It even fooled the FTL devs apparently. If you were to use other characters (like accents or fancy symbols), you *would* have issues.

Isn't it just the ship layout plain text files that use (require ?) archaic Windows "carriage returns"?

That's true, but there's no benefit in leaving the resources inconsistent as a whole, so I standardize txt line-endings the same way.

There are mods that use CR-LF.
There are mods that use LF.
And there are others using CR (old Mac style).

Hope I didn't offend you there
I just prefer not to use anything even remotely Windows-related

*shrug*

[kartoFlane]

C'mon, 600 megs!? There is no way text is being handled that inefficiently.

600 MB is 600 000K in task manager... I'm testing it now, the program uses ~35 000K before parsing, and ~44 000K after parsing. Odd... No idea how to explain the Mac spike

Pegging the CPU usually means a thread is doing an endless loop without blocking/sleeping.

So it's the main program loop. I'll take a closer look at that, I remember messing a bit with it...

Edit:
So that was pretty bad. From

Code: Select all

while (!shell.isDisposed())
    display.readAndDispatch();

to:

Code: Select all

while (!shell.isDisposed()) {
    if (!display.readAndDispatch())
        display.sleep();
}

CPU usage down from 50% at all times to 0-2% while not parsing. Whooops

This is an example where EC will throw the aforementioned error, although 595 pixels divided by 35 pixels is... 17, and not 9.

Found the culprit - the animSheet tags are repeated:

Code: Select all

<animSheet name="bomb_fire" w="595" h="50" fw="35" fh="50">weapons/bomblauncher_fire_strip17.png</animSheet>
<weaponAnim name="bomb_fire">
  <sheet>bomb_fire</sheet>
  <desc length="17" x="0" y="0"/>
  ...
</weaponAnim>

...

<animSheet name="bomb_fire" w="315" h="35" fw="35" fh="35">weapons/bomb_fire_strip9.png</animSheet>
<anim name="bomb_fire">
  <sheet>bomb_fire</sheet>
  <desc length="9" x="0" y="0"/>
  <time>0.35</time>
</anim>

Anims and weaponAnims can be distinguished from one another without a problem, but the <animSheet>s share the same name, and thus overwrite each other, and there's no way for me to distinguish them at runtime...

Ah, the importance of good naming habits. Adding "_launcher" at the end of the name of the first sheet would've fixed that...

I'm trying to narrow down where the problem is, the way SMM generates text, or the way the Error Checker is showing it.

I'm somewhat positive that it's the GUI's fault - IIRC SWT creates instances of native OS widgets, so the StyledText widget may be expecting Mac-style line endings by default...
I'm reading through the documentation now.

[ApexMods]

I'm trying to narrow down where the problem is, the way SMM generates text, or the way the Error Checker is showing it.

Could you compare EC's view with the way SMM's File-"XML Sandbox" displays, say events.xml?

Both won't display any empty lines (multiple line breaks), that's basically it. It's really not that much of a deal. Together with the (rather large) non-monospace font used for XML by EC it can be somewhat of a nuisance, especially when compared to well-structured original code.

[Vhati]

Both won't display any empty lines (multiple line breaks), that's basically it.

This is what I see in SMM's sandbox...

Within comments, there are tabs, multiple spaces after "SUMMARY", and blank lines.

Clipboard01.png

Between tags, blank lines and spaces are omitted, notably between top-level tags,
like this <textList> and <event>.

Clipboard02.png

Are you referring to the latter? I could probably add a blank back in above each top-level tag...*
Or are you seeing something different?

Your phrasing was unclear...
"stripping of all multiple line breaks (and spaces ?)"
"any formatted paragraphs of code and comments(!) into one huge wall"


* That wouldn't recreate the extra whitespace between blueprint sections however.
Hmm, come to think of it... Edit: Nah, I don't think I can preserve that whitespace.

[ApexMods]

CPU usage down from 50% at all times to 0-2% while not parsing. Whooops

Excellent! Can't wait for the next update!

Found the culprit - the animSheet tags are repeated:

...

Anims and weaponAnims can be distinguished from one another without a problem, but the <animSheet>s share the same name, and thus overwrite each other, and there's no way for me to distinguish them at runtime...

Ah, the importance of good naming habits. Adding "_launcher" at the end of the name of the first sheet would've fixed that...

Thanks for the heads-up! Stupid me, I never checked for non-unique tags in the original game.

Within comments, there are tabs, multiple spaces after "SUMMARY", and blank lines.

...

Between tags, blank lines and spaces are omitted, notably between top-level tags,
like this <textList> and <event>.

SMM works exactly as you intended at my end, only preserving e.g. multiple line breaks within comment tags and stripping them everywhere else. It's just my personal preference to visually split (with one or more empty lines) unrelated object blocks (and their comments), while keeping related objects (and their comments) in continuous tag blocks (without empty lines in between), as this makes it easier for my old eyes to read/analyze/debug e.g. events that consist of many related event and text lists.

Within EC, code readability suffers some more, due to e.g. its (or SWT's) non-monospace (non-fixed-width) font, substituting tabs with a whitespace character, etc. on top of the lost formatting caused by SMM. But, and I cannot stress this enough now: it's not a big issue at all, as I do all my editing in a separate XML editor anyway, so I just switch back and forth between EC and my editor for debugging.

[Vhati]

But, and I cannot stress this enough now: it's not a big issue at all, as I do all my editing in a separate XML editor anyway

*chuckle* I'm always lookin' for an excuse to troubleshoot. I enjoy it.
Some people wrestle Mantis boarders. I wrestle bugs.

SMM works exactly as you intended at my end

Thanks for the patience, I don't have a Mac to test on, so I was a bit starved for feedback.

[ApexMods]

OK, here's a couple of additional reports on EC.

• ship statements without explicit hostile="false" attribute are reported as critical error, although the negative boolean attribute is not mandatory/necessary ("Syntax error (missing hostile attribute)")
• some unique self-terminating tags, e.g. <store/>, are falsely reported as errors when followed by further (valid) statements, e.g. when placed at the beginning of an event block ("This tag contains a terminator tag (<event/>, <store/>) that is not last in the nested structure (<store/>)")
• textList objects loaded from within a non-event statement (e.g. inside a <ship... <surrender... block) are not recognized as referenced ("This object is not referenced from anywhere, and is not used in-game")
• new custom/mod weapon blueprints (e.g. FOO_WEAPON) loaded exclusively through a particular ship blueprint (e.g. FOO_SHIP) and not referenced anywhere else are not recognized as being used ("This object is not referenced from anywhere, and is not used in-game")

Have to say, the orphaned object detection alone makes EC absolutely indispensable! Hunting down unused/obsolete or wrongly linked elements is a breeze, thanks to your excellent work, kartoFlane!

[kartoFlane]

Thanks for the info, I've corrected the code.

However:

some unique self-terminating tags, e.g. <store/>, are falsely reported as errors when followed by further (valid) statements, e.g. when placed at the beginning of an event block ("This tag contains a terminator tag (<event/>, <store/>) that is not last in the nested structure (<store/>)")

As far as I'm aware, everything after a <store/> tag in an event won't ever be executed by the game. Thus, it acts as an event termination tag, the same as <event/>.
I'm not completely sure, but I tested this a bit, and it seemed to behave that way. No idea whether there are more tags like this though.

new custom/mod weapon blueprints (e.g. FOO_WEAPON) loaded exclusively through a particular ship blueprint (e.g. FOO_SHIP) and not referenced anywhere else are not recognized as being used ("This object is not referenced from anywhere, and is not used in-game")

That's odd, it should be working. Does it happen with drones, too? What about blueprint lists linked via <weaponList load="FOO"/>, and their contents?
Or is it just the

Code: Select all

<weaponList count="#">
  <weapon name="FOO_WEAPON"/>
  ...
</weaponList>

that is problematic?